-- Logs begin at Mon 2022-07-18 15:47:22 UTC, end at Mon 2022-07-18 17:37:03 UTC. -- Jul 18 16:22:58.820594 dv-cinder-846172-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 18 16:22:58.838371 dv-cinder-846172-3-iscsi devstack@keystone.service[80357]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 18 16:22:58.838371 dv-cinder-846172-3-iscsi devstack@keystone.service[80357]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Jul 18 16:22:58 2022] *** Jul 18 16:22:58.838371 dv-cinder-846172-3-iscsi devstack@keystone.service[80357]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 18 16:22:58.838371 dv-cinder-846172-3-iscsi devstack@keystone.service[80357]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 18 16:22:58.838371 dv-cinder-846172-3-iscsi devstack@keystone.service[80357]: nodename: dv-cinder-846172-3-iscsi Jul 18 16:22:58.838371 dv-cinder-846172-3-iscsi devstack@keystone.service[80357]: machine: x86_64 Jul 18 16:22:58.838371 dv-cinder-846172-3-iscsi devstack@keystone.service[80357]: clock source: unix Jul 18 16:22:58.838371 dv-cinder-846172-3-iscsi devstack@keystone.service[80357]: pcre jit disabled Jul 18 16:22:58.838371 dv-cinder-846172-3-iscsi devstack@keystone.service[80357]: detected number of CPU cores: 4 Jul 18 16:22:58.838371 dv-cinder-846172-3-iscsi devstack@keystone.service[80357]: current working directory: / Jul 18 16:22:58.838371 dv-cinder-846172-3-iscsi devstack@keystone.service[80357]: detected binary path: /usr/bin/uwsgi-core Jul 18 16:22:58.838371 dv-cinder-846172-3-iscsi devstack@keystone.service[80357]: your processes number limit is 47680 Jul 18 16:22:58.838371 dv-cinder-846172-3-iscsi devstack@keystone.service[80357]: your memory page size is 4096 bytes Jul 18 16:22:58.838371 dv-cinder-846172-3-iscsi devstack@keystone.service[80357]: detected max file descriptor number: 2048 Jul 18 16:22:58.838371 dv-cinder-846172-3-iscsi devstack@keystone.service[80357]: lock engine: pthread robust mutexes Jul 18 16:22:58.838371 dv-cinder-846172-3-iscsi devstack@keystone.service[80357]: thunder lock: enabled Jul 18 16:22:58.838371 dv-cinder-846172-3-iscsi devstack@keystone.service[80357]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 18 16:22:58.838371 dv-cinder-846172-3-iscsi devstack@keystone.service[80357]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 18 16:22:58.874831 dv-cinder-846172-3-iscsi devstack@keystone.service[80357]: Python main interpreter initialized at 0x55d17552b9c0 Jul 18 16:22:58.874831 dv-cinder-846172-3-iscsi devstack@keystone.service[80357]: python threads support enabled Jul 18 16:22:58.874831 dv-cinder-846172-3-iscsi devstack@keystone.service[80357]: your server socket listen backlog is limited to 100 connections Jul 18 16:22:58.874831 dv-cinder-846172-3-iscsi devstack@keystone.service[80357]: your mercy for graceful operations on workers is 90 seconds Jul 18 16:22:58.875205 dv-cinder-846172-3-iscsi devstack@keystone.service[80357]: mapped 403077 bytes (393 KB) for 2 cores Jul 18 16:22:58.875278 dv-cinder-846172-3-iscsi devstack@keystone.service[80357]: *** Operational MODE: preforking *** Jul 18 16:22:58.875583 dv-cinder-846172-3-iscsi devstack@keystone.service[80357]: *** uWSGI is running in multiple interpreter mode *** Jul 18 16:22:58.875583 dv-cinder-846172-3-iscsi devstack@keystone.service[80357]: spawned uWSGI master process (pid: 80357) Jul 18 16:22:58.875865 dv-cinder-846172-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 18 16:22:58.883517 dv-cinder-846172-3-iscsi devstack@keystone.service[80357]: spawned uWSGI worker 1 (pid: 80367, cores: 1) Jul 18 16:22:58.883517 dv-cinder-846172-3-iscsi devstack@keystone.service[80357]: spawned uWSGI worker 2 (pid: 80368, cores: 1) Jul 18 16:22:58.883517 dv-cinder-846172-3-iscsi devstack@keystone.service[80357]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 18 16:23:04.456215 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 18 16:23:04.456215 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: from cryptography.utils import int_from_bytes Jul 18 16:23:04.457305 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 18 16:23:04.457305 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: from cryptography.utils import int_from_bytes Jul 18 16:23:04.459877 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 18 16:23:04.459877 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: from cryptography.utils import int_from_bytes Jul 18 16:23:04.460370 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 18 16:23:04.460370 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: from cryptography.utils import int_from_bytes Jul 18 16:23:04.507581 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 18 16:23:04.507845 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 18 16:23:04.507977 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] command line args: [] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 18 16:23:04.508870 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 18 16:23:04.509043 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ================================================================================ {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 18 16:23:04.509305 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] admin_token = **** {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.509558 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] config_dir = [] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.509748 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.509930 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] config_source = [] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.510112 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] debug = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.510313 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.511717 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.511717 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.511717 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.511717 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.511717 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] list_limit = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.511717 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] log_config_append = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.512381 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.512510 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 18 16:23:04.512595 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] log_dir = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.512667 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 18 16:23:04.512794 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] log_file = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.512871 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] command line args: [] {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 18 16:23:04.512992 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.513068 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 18 16:23:04.513182 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.513256 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ================================================================================ {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 18 16:23:04.513382 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.513458 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] admin_token = **** {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.513579 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.513657 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] config_dir = [] {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.513771 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.513855 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.513928 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.514046 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] config_source = [] {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.514127 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.514205 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] debug = True {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.514205 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.514474 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.514677 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.514759 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.514884 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.515009 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.515145 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.515269 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.515269 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.515550 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.515550 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] list_limit = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.515887 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.515974 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] log_config_append = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.516053 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.516172 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.516172 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] log_dir = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.516388 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.516527 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] log_file = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.516527 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.516765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.26/identity {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.516895 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.517032 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] publish_errors = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.517172 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.517258 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.517336 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.517463 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.517539 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.517618 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.517703 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.517776 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.517852 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.517924 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.517999 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.518124 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] standard_threads = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.518208 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.518284 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.518355 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.518424 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.518494 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.518571 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.518647 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.518717 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] use_journal = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.518717 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] use_json = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.518841 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.518929 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] use_stderr = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.519008 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.519079 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] use_syslog = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.519079 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.519204 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.519281 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.519431 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.519532 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.26/identity {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.519659 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] publish_errors = False {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.519753 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.519891 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.519969 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.520118 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.520196 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.520401 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.520605 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.520691 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.520765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.520836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.520906 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.520981 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] standard_threads = False {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.521108 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.521189 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.521358 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.521443 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.521588 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.521665 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.521747 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] use_journal = False {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.521826 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.521910 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] use_json = False {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.522065 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.522150 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] use_stderr = False {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.522276 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] use_syslog = False {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.522359 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.522454 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:23:04.522532 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.522709 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.522789 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.522949 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.523036 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.523208 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.523287 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.523446 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.523545 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.523745 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.523845 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.523980 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.524059 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.524233 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.524342 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.524465 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.524553 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.524711 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.524796 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.524956 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.525034 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.525200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.525288 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.525430 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.525509 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.525660 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.525736 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.525885 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.526012 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.526132 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.526212 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.526379 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.526457 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.526611 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.526691 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.526842 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.526925 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.527089 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.527167 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.527331 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.527410 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.527588 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.527671 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.527844 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.527921 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.528111 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.528190 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.528359 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.528436 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.528608 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.528688 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.529214 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.529300 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.529373 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.529443 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.529521 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.529599 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.529719 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.529817 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.530689 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.530790 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.530909 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.531043 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.531128 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.531201 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.531324 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.531407 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.531484 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.531615 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.531749 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.531867 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.531942 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.532119 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.532236 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.532334 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.532448 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.532528 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.532598 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.532598 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.532736 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.532813 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.532923 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.533012 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.533086 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.533162 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.533234 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.533336 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.533423 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.533505 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.533578 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.533649 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.533757 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.533837 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.533909 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.533988 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.534061 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.534165 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.534243 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.534324 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.534421 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.534509 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.534669 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.534749 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.534901 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.534980 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.535135 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.535251 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.535369 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.535455 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.535625 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.535708 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.535859 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.540906 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.541028 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.541145 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.541228 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.541289 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.541352 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.541433 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.541506 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.541578 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.541649 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.541720 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.541791 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.541872 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.542025 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.542102 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.542174 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.542251 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.542324 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.542403 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.542477 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.542548 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.542625 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.542704 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.542776 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.542860 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.542941 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.543021 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.543094 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.543166 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.543166 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.543291 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.543371 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.543449 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.543556 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.543631 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.543702 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.543785 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.543866 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.543945 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.544018 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.544095 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.544168 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.544239 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.544318 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.544391 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.544468 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.544541 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.544612 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.544683 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.544765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.544839 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.544910 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.544987 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.545060 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.545131 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.545221 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.545296 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.545368 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.545440 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.545511 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.545583 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.545662 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.545736 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.545831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.545906 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.545977 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.546048 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.546127 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.546207 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.546280 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] auth.external = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.546357 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.546430 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.546501 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.546580 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.546654 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.546725 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.546795 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.546871 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] auth.password = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.546943 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.547022 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] auth.token = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.547095 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.547165 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.547241 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.547314 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.547385 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.547463 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.547658 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.547785 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.547871 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.547945 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.548018 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.548089 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.548162 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.548242 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.548316 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.548387 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] credential.caching = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.548465 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] auth.external = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.548538 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.548610 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.548689 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.548763 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.548855 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.548938 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.549011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.549083 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] auth.password = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.549163 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.549237 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] auth.token = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.549309 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.549380 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.549452 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.549523 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.549602 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.549676 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.549748 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.549819 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.549891 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.549962 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.550042 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.550116 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.550116 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.550246 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.550319 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] credential.caching = True {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.550390 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.550474 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.550555 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.550719 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.550803 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.550976 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.551067 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] federation.caching = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.551214 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.551288 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.551444 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.551557 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.551702 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.551781 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.551950 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.552031 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.552199 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.552279 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.552433 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.552533 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.552686 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.552772 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.552922 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.553001 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.553177 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.553266 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.553414 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.553494 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.553662 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.553744 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.553900 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] identity.caching = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.553988 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.554135 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] federation.caching = True {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.554214 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.554370 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.554450 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.554606 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.554693 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.554844 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.555632 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.555730 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.555803 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.555969 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.556063 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.556210 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.556290 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.556462 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.556550 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.556716 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.556805 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.556948 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.557029 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.557180 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.557260 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.557412 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.557504 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.557658 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.557739 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] identity.caching = True {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.557906 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.557988 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.558151 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.558240 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.558400 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.558481 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.558636 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.558716 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.558908 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.559150 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.559389 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.559660 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.559902 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.560132 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.560456 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.560566 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.560739 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.560823 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.560983 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.561065 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.561238 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.561328 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.561473 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.561554 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.561724 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.561804 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.561972 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.562060 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.562207 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.562415 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.562530 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.562621 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.562698 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.562782 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.562896 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.563050 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.563132 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.563313 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.563401 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.563593 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.563675 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.563830 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.563907 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.564078 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.564176 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.564319 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.564406 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.564549 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.564632 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.564788 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.564875 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.565020 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.565099 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.565276 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.565359 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.565488 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.565575 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.565741 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.565820 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.565989 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.566066 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.566222 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.566309 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.566456 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.566540 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.566693 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.566771 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.566925 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.567012 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.567167 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.567253 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.567411 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.567508 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.567665 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.567756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.567901 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.567980 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.568154 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.568262 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.568406 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.568495 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.568640 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.568720 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.568872 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.568950 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.569108 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.569194 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.569343 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.569422 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.569583 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.569662 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.569814 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.569907 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.570064 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.570142 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.570311 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.570389 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.570586 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.571279 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.571485 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.571935 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.572010 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.572095 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.572175 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.572257 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.572330 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.572401 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.572472 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.572544 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.572670 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ldap.user = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.572759 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.572912 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.573000 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.573174 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.573254 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.573409 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.573497 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.573668 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.573746 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.573938 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.574015 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.574170 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.574282 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.574405 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.574560 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.574672 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.574749 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.574860 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.574950 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.575095 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.575175 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.575343 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.575421 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.575608 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.585427 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.585581 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.585660 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.585731 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.585802 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.585872 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] resource.caching = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.586026 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.586102 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.586205 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.586340 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.586451 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.586867 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.587030 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.587106 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.587184 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.587258 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.587329 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.587399 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.587477 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.587604 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.587676 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.587747 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.587818 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.587889 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.587977 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.588052 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] role.caching = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.588122 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.588192 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] role.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.588262 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.588332 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.588411 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.588482 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.588552 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.588647 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.588741 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.588821 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.588895 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.588965 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.589036 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.589107 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.589178 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.589257 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.589330 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.589401 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.589477 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.589549 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.589619 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.589697 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.589770 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.589846 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.589918 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] resource.caching = True {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.589989 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.590059 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.590137 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.590209 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.590209 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.590336 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.590407 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.590477 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.590555 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.590628 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.590698 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.590768 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.590839 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.590909 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.590987 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.591059 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.591130 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.591206 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.591278 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] role.caching = True {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.591348 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.591460 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] role.driver = sql {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.591460 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.591603 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.591663 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.591733 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.591803 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.591886 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.591886 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.592017 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.592087 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.592158 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.592229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.592307 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.592380 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.592450 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.592520 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.592590 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.592660 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.592737 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.592809 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.592880 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.592939 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.592996 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.593073 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.593144 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.593215 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] token.caching = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.593284 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.593354 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.593424 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.593501 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.593501 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.593626 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.593626 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.593742 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.593811 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.593884 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.593884 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.594001 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.594070 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.594140 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.594210 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.594286 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.594286 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.594410 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.594410 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.594526 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.594603 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.594682 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.594754 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.594818 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.594818 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.594933 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.595002 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.595080 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.595151 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.595222 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.595302 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.595373 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.595443 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.595604 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.595682 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.595753 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.connection = **** {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.595837 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.595919 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.595992 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.596067 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.596147 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.596219 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.596290 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] token.caching = True {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.596360 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.596430 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.596500 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.596578 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.596650 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.596721 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.596790 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.596850 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.596907 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.597025 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.597100 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.597171 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.597240 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.597311 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.597381 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.597459 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.597531 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.597602 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.597673 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.597745 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.597815 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.597893 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.597965 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.598036 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.598106 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.598176 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.598246 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.598324 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.598396 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.598396 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 18 16:23:04.598524 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.598524 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.598524 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.598524 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] database.connection = **** {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.598524 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.598524 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.598524 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.598524 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.598524 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.599142 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.599142 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.599142 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.599142 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.599142 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.599142 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.599516 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.599610 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.599847 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.600089 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.600321 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.600578 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.600794 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:23:04.600962 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 18 16:23:04.628207 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.628433 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.628659 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.630817 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.631048 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.631275 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.639917 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.640492 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.669002 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.669228 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.669721 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.669954 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.821307 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.822359 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.872271 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.872505 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.876501 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.876912 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.938989 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.939196 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.939420 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.939671 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.939892 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.940107 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.941332 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.941564 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.941787 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.942005 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.942222 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.942435 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.966266 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.966520 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.966653 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.966780 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.966908 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.967050 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.967177 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.967315 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.967441 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.967605 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.967731 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.967868 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.967995 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.968121 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.978915 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.979572 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.991920 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.992149 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.992361 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.993038 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.993170 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.993371 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.993458 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.993522 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.993582 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.993704 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.993801 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.994017 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.994227 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:04.994437 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:23:05.011793 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.011982 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.012126 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 18 16:23:05.012253 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 18 16:23:05.012378 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.012508 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.012632 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 18 16:23:05.012771 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 18 16:23:05.012901 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.013043 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.013172 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.013301 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.013429 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.013567 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.013717 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.013842 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.025026 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.025219 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.025348 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.025489 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.025621 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.025764 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.025894 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.026025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.033978 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:23:05.034118 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80367) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:23:05.037272 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:23:05.037406 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80367) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:23:05.037558 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.037698 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.037874 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.038012 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.038472 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.038600 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.038759 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.038888 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.039013 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.039139 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.039265 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.039408 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.039654 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.039730 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.039935 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.040062 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.040211 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.040314 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.040479 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.040561 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.067389 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.067747 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.069069 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80367) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:23:05.069344 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.069427 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:23:05.069653 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.075338 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80367) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:23:05.075511 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:23:05.075624 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.075790 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.075877 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.076066 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.083114 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80367) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:23:05.083360 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:23:05.083446 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.083675 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.088127 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80367) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:23:05.088363 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:23:05.088466 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.088653 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.088738 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.088934 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.089016 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.089255 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.089356 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.089487 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.089569 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.089751 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.089833 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.090019 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.090099 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.090285 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.109036 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.109172 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.109335 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.109414 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.113856 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:23:05.113961 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80367) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:23:05.118050 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.118195 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.118341 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.118424 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.123337 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:23:05.123457 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80367) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:23:05.127046 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:23:05.127195 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80367) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:23:05.130915 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.131216 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.131513 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.131802 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.132102 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.132396 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.134005 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.134317 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.134601 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.134896 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.135369 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.135917 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.152952 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:23:05.153357 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.153627 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.153896 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.159504 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80367) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:23:05.159904 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.160176 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.160472 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.161842 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.163834 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.164109 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.167751 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:23:05.168048 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.168322 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.168607 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.168888 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.169323 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.171030 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.171336 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.174381 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80367) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:23:05.174658 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.174934 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.175228 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.175529 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.182642 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:23:05.182917 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.183190 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.183464 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.183767 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.184052 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.184322 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.190938 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80367) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:23:05.191215 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.191508 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.191791 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.192080 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.192348 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.192625 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.203727 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:23:05.206990 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:23:05.210325 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.210773 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80367) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:23:05.212633 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.214509 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:23:05.214665 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80367) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:23:05.214810 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.215050 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.217933 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.219533 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.221211 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80367) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:23:05.221480 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.221763 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.222541 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.224091 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:23:05.237424 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.237424 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.237424 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.237424 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.237680 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.237680 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80367) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:23:05.237680 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.237680 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.237680 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.237680 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.238112 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:23:05.238376 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.238651 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.246417 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80367) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:23:05.246707 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.246995 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.247080 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:23:05.247362 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.247666 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.247938 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.248223 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.248501 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.248771 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.249037 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.249309 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.249574 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.249847 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.250119 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.250381 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.250647 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.254856 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80367) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:23:05.255132 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.255410 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.255704 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.255971 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.256245 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.256518 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.256787 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.257055 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.257316 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.257587 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.257846 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.258719 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.259024 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:23:05.289221 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55d17552b9c0 pid: 80368 (default app) Jul 18 16:23:05.292739 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c57fb-be40-4769-adb5-431ad5977a9a None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:05.293031 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c57fb-be40-4769-adb5-431ad5977a9a None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:05.293318 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c57fb-be40-4769-adb5-431ad5977a9a None None] PATH_INFO: `/v3` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:05.297570 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1/1] 10.222.0.26 () {50 vars in 791 bytes} [Mon Jul 18 16:23:05 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:05.297813 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55d17552b9c0 pid: 80367 (default app) Jul 18 16:23:13.938532 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccabb81-c07a-4cda-b3d1-47077b75b782 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:13.939096 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccabb81-c07a-4cda-b3d1-47077b75b782 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:13.939096 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccabb81-c07a-4cda-b3d1-47077b75b782 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:13.941042 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 2/2] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:23:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:13.948579 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36024af0-d11e-4cfe-9df3-e550b5068b51 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:13.952432 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36024af0-d11e-4cfe-9df3-e550b5068b51 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:13.952432 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36024af0-d11e-4cfe-9df3-e550b5068b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:14.020631 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: INFO dbcounter [None req-36024af0-d11e-4cfe-9df3-e550b5068b51 None None] Registered counter for database keystone Jul 18 16:23:14.043634 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG oslo_db.sqlalchemy.engines [None req-36024af0-d11e-4cfe-9df3-e550b5068b51 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80367) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 18 16:23:14.050007 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writer thread running {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 18 16:23:14.228662 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG passlib.handlers.bcrypt [None req-36024af0-d11e-4cfe-9df3-e550b5068b51 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80367) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 18 16:23:14.229058 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG passlib.handlers.bcrypt [None req-36024af0-d11e-4cfe-9df3-e550b5068b51 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80367) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 18 16:23:14.286266 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-36024af0-d11e-4cfe-9df3-e550b5068b51 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:14.288621 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-36024af0-d11e-4cfe-9df3-e550b5068b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:14.418636 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1/3] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:13 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 474 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 18 16:23:14.571321 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccabb81-c07a-4cda-b3d1-47077b75b782 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:14.640628 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: INFO dbcounter [None req-8ccabb81-c07a-4cda-b3d1-47077b75b782 None None] Registered counter for database keystone Jul 18 16:23:14.660534 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG oslo_db.sqlalchemy.engines [None req-8ccabb81-c07a-4cda-b3d1-47077b75b782 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80368) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 18 16:23:14.666956 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writer thread running {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 18 16:23:14.784907 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccabb81-c07a-4cda-b3d1-47077b75b782 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:14.789438 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d1902-146f-4130-bc5e-fc49cacd29e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:14.790111 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d1902-146f-4130-bc5e-fc49cacd29e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:14.790394 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d1902-146f-4130-bc5e-fc49cacd29e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:14.790668 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d1902-146f-4130-bc5e-fc49cacd29e2 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:14.798789 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5d1902-146f-4130-bc5e-fc49cacd29e2 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:14.853947 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: WARNING keystone.common.rbac_enforcer.enforcer [None req-5e5d1902-146f-4130-bc5e-fc49cacd29e2 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 18 16:23:14.854286 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5d1902-146f-4130-bc5e-fc49cacd29e2 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:14.864596 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 3/4] 10.222.0.26 () {58 vars in 1324 bytes} [Mon Jul 18 16:23:14 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 16:23:14.871374 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36024af0-d11e-4cfe-9df3-e550b5068b51 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:14.888841 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36024af0-d11e-4cfe-9df3-e550b5068b51 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:14.893281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4e077d-f0b6-4486-99e0-01903c7408e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:14.894408 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e077d-f0b6-4486-99e0-01903c7408e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:14.894698 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e077d-f0b6-4486-99e0-01903c7408e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:14.894977 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e077d-f0b6-4486-99e0-01903c7408e9 admin admin] PATH_INFO: `/v3/projects/9e08d80b293148beac001ab7796d9842` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:14.896659 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4e077d-f0b6-4486-99e0-01903c7408e9 admin admin] RBAC: Authorizing `identity:get_project(project_id=9e08d80b293148beac001ab7796d9842)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:14.948090 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ae4e077d-f0b6-4486-99e0-01903c7408e9 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 18 16:23:14.948441 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4e077d-f0b6-4486-99e0-01903c7408e9 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:14.953041 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 2/5] 10.222.0.26 () {56 vars in 1207 bytes} [Mon Jul 18 16:23:14 2022] GET /identity/v3/projects/9e08d80b293148beac001ab7796d9842 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:16.976476 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d934bfb2-09df-4c4b-ae70-7a1992a0edc8 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:16.977066 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d934bfb2-09df-4c4b-ae70-7a1992a0edc8 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:16.977309 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d934bfb2-09df-4c4b-ae70-7a1992a0edc8 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:16.981505 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 4/6] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:23:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:23:16.986904 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae16e92-4635-4641-bf5f-0320787692de None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:16.987327 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae16e92-4635-4641-bf5f-0320787692de None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:16.987468 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae16e92-4635-4641-bf5f-0320787692de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:17.006464 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-0ae16e92-4635-4641-bf5f-0320787692de None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:17.008660 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-0ae16e92-4635-4641-bf5f-0320787692de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:17.016670 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 3/7] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:16 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 18 16:23:17.038111 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d934bfb2-09df-4c4b-ae70-7a1992a0edc8 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:17.051141 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d934bfb2-09df-4c4b-ae70-7a1992a0edc8 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:17.055939 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87442cf3-2ae7-4f9d-a325-92725d0073e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:17.056578 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87442cf3-2ae7-4f9d-a325-92725d0073e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:17.056864 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87442cf3-2ae7-4f9d-a325-92725d0073e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:17.057140 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87442cf3-2ae7-4f9d-a325-92725d0073e6 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:17.065052 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87442cf3-2ae7-4f9d-a325-92725d0073e6 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:17.066853 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87442cf3-2ae7-4f9d-a325-92725d0073e6 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:17.076222 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 5/8] 10.222.0.26 () {58 vars in 1324 bytes} [Mon Jul 18 16:23:17 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 16:23:17.081888 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae16e92-4635-4641-bf5f-0320787692de None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:17.091066 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae16e92-4635-4641-bf5f-0320787692de None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:17.095373 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46fa981-38cf-465c-bfd5-bb2dda74655a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:17.096273 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46fa981-38cf-465c-bfd5-bb2dda74655a admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:17.096558 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46fa981-38cf-465c-bfd5-bb2dda74655a admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:17.096831 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46fa981-38cf-465c-bfd5-bb2dda74655a admin admin] PATH_INFO: `/v3/users/e18df28d16ba462694258c7ad963170a` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:17.098239 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46fa981-38cf-465c-bfd5-bb2dda74655a admin admin] RBAC: Authorizing `identity:get_user(user_id=e18df28d16ba462694258c7ad963170a)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:17.100104 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46fa981-38cf-465c-bfd5-bb2dda74655a admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:17.101733 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 4/9] 10.222.0.26 () {56 vars in 1198 bytes} [Mon Jul 18 16:23:17 2022] GET /identity/v3/users/e18df28d16ba462694258c7ad963170a => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:19.218089 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2407f2-056a-4817-99cc-af6510528b9e None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:19.218606 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2407f2-056a-4817-99cc-af6510528b9e None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:19.218606 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2407f2-056a-4817-99cc-af6510528b9e None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:19.219945 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 6/10] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:23:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:23:19.225130 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f168239-dbd1-408e-b8c6-efb53f3adad3 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:19.225556 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f168239-dbd1-408e-b8c6-efb53f3adad3 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:19.225819 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f168239-dbd1-408e-b8c6-efb53f3adad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:19.244066 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-0f168239-dbd1-408e-b8c6-efb53f3adad3 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:19.245282 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-0f168239-dbd1-408e-b8c6-efb53f3adad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:19.261734 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce14b5-ef33-47e8-aa13-7c2d3d3a5547 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:19.262032 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce14b5-ef33-47e8-aa13-7c2d3d3a5547 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:19.262313 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce14b5-ef33-47e8-aa13-7c2d3d3a5547 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:19.263752 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 7/11] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:23:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:23:19.269038 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a16280-3593-40c1-bfdc-cca2626df326 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:19.269329 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a16280-3593-40c1-bfdc-cca2626df326 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:19.269612 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a16280-3593-40c1-bfdc-cca2626df326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:19.310302 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG passlib.handlers.bcrypt [None req-e8a16280-3593-40c1-bfdc-cca2626df326 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80368) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 18 16:23:19.310728 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG passlib.handlers.bcrypt [None req-e8a16280-3593-40c1-bfdc-cca2626df326 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80368) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 18 16:23:19.328252 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 5/12] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:19.352524 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f168239-dbd1-408e-b8c6-efb53f3adad3 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:19.357637 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-e8a16280-3593-40c1-bfdc-cca2626df326 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:19.359267 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e8a16280-3593-40c1-bfdc-cca2626df326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:19.385485 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:19.462864 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f168239-dbd1-408e-b8c6-efb53f3adad3 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:19.470610 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 8/13] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:23:19.493682 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a16280-3593-40c1-bfdc-cca2626df326 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:19.496840 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0716e6-716b-4413-8c87-9a45a56d7d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:19.497648 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0716e6-716b-4413-8c87-9a45a56d7d7f None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:19.497929 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0716e6-716b-4413-8c87-9a45a56d7d7f None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:19.498205 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0716e6-716b-4413-8c87-9a45a56d7d7f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:19.509835 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0716e6-716b-4413-8c87-9a45a56d7d7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:19.521115 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-7a0716e6-716b-4413-8c87-9a45a56d7d7f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:23:19.521115 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:19.521115 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:19.521115 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:19.521115 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:19.521115 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:19.521115 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:19.521115 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:19.521115 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:19.521115 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:19.521115 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:19.521115 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:19.521115 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:19.521115 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:19.521115 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:19.521115 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:19.522053 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:19.522053 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:19.522053 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:19.522053 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:19.522053 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:19.522053 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:19.522053 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:19.522053 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:19.522053 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:19.522053 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:19.522053 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:19.522053 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:19.522053 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:19.522053 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:19.522053 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:19.522053 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:19.522956 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:19.522956 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:19.522956 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:19.522956 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:19.522956 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:19.522956 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:19.522956 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:19.522956 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:19.522956 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:19.522956 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:19.522956 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:23:19.522956 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:23:19.522956 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 6/14] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 18 16:23:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:23:19.529310 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0716e6-716b-4413-8c87-9a45a56d7d7f None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:19.594990 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a16280-3593-40c1-bfdc-cca2626df326 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:19.597348 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:19.620871 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:19.632481 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd2199-c0e7-4189-a141-271b4573907d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:19.634046 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd2199-c0e7-4189-a141-271b4573907d None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:19.634337 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd2199-c0e7-4189-a141-271b4573907d None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:19.634615 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd2199-c0e7-4189-a141-271b4573907d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:19.639409 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0716e6-716b-4413-8c87-9a45a56d7d7f None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:19.674021 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3920b3-f864-4a7d-83bf-92242edf4012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:19.674746 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3920b3-f864-4a7d-83bf-92242edf4012 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:19.675279 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3920b3-f864-4a7d-83bf-92242edf4012 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:19.676124 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3920b3-f864-4a7d-83bf-92242edf4012 None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:19.676888 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3920b3-f864-4a7d-83bf-92242edf4012 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:19.677192 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3920b3-f864-4a7d-83bf-92242edf4012 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:19.678592 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3920b3-f864-4a7d-83bf-92242edf4012 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:19.690895 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 7/15] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:23:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:23:19.696409 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3920b3-f864-4a7d-83bf-92242edf4012 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:19.702612 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debd2199-c0e7-4189-a141-271b4573907d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:19.704276 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debd2199-c0e7-4189-a141-271b4573907d None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:19.796917 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3920b3-f864-4a7d-83bf-92242edf4012 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:19.803270 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 9/16] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:23:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:23:19.809835 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd2199-c0e7-4189-a141-271b4573907d None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:19.812759 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:19.836850 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1369b02c-a296-41e9-8769-d48b82333d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:19.837871 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1369b02c-a296-41e9-8769-d48b82333d64 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:19.838150 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1369b02c-a296-41e9-8769-d48b82333d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:19.838428 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1369b02c-a296-41e9-8769-d48b82333d64 None admin] PATH_INFO: `/v3/users/e18df28d16ba462694258c7ad963170a` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:19.839685 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1369b02c-a296-41e9-8769-d48b82333d64 None admin] RBAC: Authorizing `identity:get_user(user_id=e18df28d16ba462694258c7ad963170a)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:19.841253 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1369b02c-a296-41e9-8769-d48b82333d64 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:19.842889 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 8/17] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:19 2022] GET /identity/v3/users/e18df28d16ba462694258c7ad963170a => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:19.848299 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1369b02c-a296-41e9-8769-d48b82333d64 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:19.863700 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:19.910876 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd2199-c0e7-4189-a141-271b4573907d None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:19.945296 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef3ffa6-944d-44ad-9580-bdbe44ea7d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:19.946262 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef3ffa6-944d-44ad-9580-bdbe44ea7d11 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:19.946546 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef3ffa6-944d-44ad-9580-bdbe44ea7d11 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:19.946821 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef3ffa6-944d-44ad-9580-bdbe44ea7d11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:19.949122 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1369b02c-a296-41e9-8769-d48b82333d64 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:19.968149 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef3ffa6-944d-44ad-9580-bdbe44ea7d11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:19.969650 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef3ffa6-944d-44ad-9580-bdbe44ea7d11 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:19.985146 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015f7683-59d7-4b16-b235-f311ceec3ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:19.985949 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f7683-59d7-4b16-b235-f311ceec3ac5 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:19.986238 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f7683-59d7-4b16-b235-f311ceec3ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:19.986513 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f7683-59d7-4b16-b235-f311ceec3ac5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:19.987733 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015f7683-59d7-4b16-b235-f311ceec3ac5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:19.989199 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015f7683-59d7-4b16-b235-f311ceec3ac5 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:19.990811 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 9/18] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:23:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:19.991006 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-bef3ffa6-944d-44ad-9580-bdbe44ea7d11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:23:19.991006 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:19.991006 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:19.991006 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:19.991006 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:19.991006 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:19.991006 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:19.991006 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:19.991006 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:19.991006 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:19.991006 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:19.991006 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:19.991006 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:23:19.991006 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:23:19.991006 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:23:19.991896 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:23:19.991896 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:19.991896 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:19.991896 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:19.991896 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:19.991896 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:19.991896 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:19.991896 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:19.991896 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:19.991896 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:19.991896 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:19.991896 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:19.991896 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:19.991896 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:19.991896 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:19.991896 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:19.991896 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:19.993293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:23:19.993293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:23:19.993293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:23:19.993293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:23:19.993293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 10/19] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:23:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:23:19.997583 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015f7683-59d7-4b16-b235-f311ceec3ac5 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:19.998436 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef3ffa6-944d-44ad-9580-bdbe44ea7d11 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:20.061787 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:20.085040 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:20.106734 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015f7683-59d7-4b16-b235-f311ceec3ac5 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:20.107755 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef3ffa6-944d-44ad-9580-bdbe44ea7d11 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:20.140587 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0295c0a-8dc4-4b59-ac35-42a00f31c7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:20.141253 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0295c0a-8dc4-4b59-ac35-42a00f31c7d9 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:20.141539 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0295c0a-8dc4-4b59-ac35-42a00f31c7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:20.141817 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0295c0a-8dc4-4b59-ac35-42a00f31c7d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:20.142659 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0295c0a-8dc4-4b59-ac35-42a00f31c7d9 None admin] RBAC: Adding query filter params (role.id=98eaadb3669f4b608d572c001742f559, scope.domain.id=default, user.id=e18df28d16ba462694258c7ad963170a) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:20.142969 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0295c0a-8dc4-4b59-ac35-42a00f31c7d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:20.143343 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c909e-8b81-419b-9555-467501489fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:20.144044 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c909e-8b81-419b-9555-467501489fab None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:20.144335 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c909e-8b81-419b-9555-467501489fab None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:20.144457 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0295c0a-8dc4-4b59-ac35-42a00f31c7d9 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:20.144616 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c909e-8b81-419b-9555-467501489fab None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:20.145973 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462c909e-8b81-419b-9555-467501489fab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:20.145973 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462c909e-8b81-419b-9555-467501489fab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:20.147026 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462c909e-8b81-419b-9555-467501489fab None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:20.152481 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 10/20] 10.222.0.26 () {56 vars in 1322 bytes} [Mon Jul 18 16:23:19 2022] GET /identity/v3/role_assignments?user.id=e18df28d16ba462694258c7ad963170a&scope.domain.id=default&role.id=98eaadb3669f4b608d572c001742f559 => generated 233 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:20.167923 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 11/21] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:23:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:23:22.467053 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce0b018-cf5f-4246-9582-313296e6a732 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:22.468124 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce0b018-cf5f-4246-9582-313296e6a732 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:22.468628 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce0b018-cf5f-4246-9582-313296e6a732 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:22.470511 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 11/22] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:23:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:22.477393 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d85f5-18cb-4c66-b275-3eabfaf3ca1e None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:22.478306 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d85f5-18cb-4c66-b275-3eabfaf3ca1e None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:22.478806 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d85f5-18cb-4c66-b275-3eabfaf3ca1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:22.504579 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-d55d85f5-18cb-4c66-b275-3eabfaf3ca1e None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:22.505822 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d55d85f5-18cb-4c66-b275-3eabfaf3ca1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:22.592967 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 12/23] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:22.637022 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce0b018-cf5f-4246-9582-313296e6a732 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:22.687083 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208e65d-27a1-4059-96d5-72a6466db10c None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:22.687866 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208e65d-27a1-4059-96d5-72a6466db10c None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:22.688800 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208e65d-27a1-4059-96d5-72a6466db10c None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:22.691089 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 13/24] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:23:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:23:22.696985 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd85bab-b0e5-4a20-9d8c-afed401966a3 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:22.697577 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd85bab-b0e5-4a20-9d8c-afed401966a3 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:22.698065 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd85bab-b0e5-4a20-9d8c-afed401966a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:22.719811 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-3dd85bab-b0e5-4a20-9d8c-afed401966a3 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:22.721349 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-3dd85bab-b0e5-4a20-9d8c-afed401966a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:22.783633 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:22.822315 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce0b018-cf5f-4246-9582-313296e6a732 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:22.827716 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 14/25] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:22.829441 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b437f0-d05e-4ece-b424-40d88d39e22a None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:22.829960 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b437f0-d05e-4ece-b424-40d88d39e22a None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:22.830447 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b437f0-d05e-4ece-b424-40d88d39e22a None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:22.832468 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 15/26] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:23:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:22.845038 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e4c8ba-5bbe-4917-b200-516e900484a9 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:22.845327 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e4c8ba-5bbe-4917-b200-516e900484a9 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:22.845604 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e4c8ba-5bbe-4917-b200-516e900484a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:22.882250 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-60e4c8ba-5bbe-4917-b200-516e900484a9 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:22.884872 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-60e4c8ba-5bbe-4917-b200-516e900484a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:22.905012 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c11de1-ebb3-46e1-b7dc-13b0cb6427ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:22.905827 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c11de1-ebb3-46e1-b7dc-13b0cb6427ac None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:22.906111 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c11de1-ebb3-46e1-b7dc-13b0cb6427ac None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:22.906386 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c11de1-ebb3-46e1-b7dc-13b0cb6427ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:22.930997 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c11de1-ebb3-46e1-b7dc-13b0cb6427ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:22.938840 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-65c11de1-ebb3-46e1-b7dc-13b0cb6427ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:23:22.938840 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:22.938840 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:22.938840 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:22.938840 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:22.938840 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:22.938840 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:22.938840 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:22.938840 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:22.938840 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:22.938840 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:22.938840 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:22.938840 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:22.938840 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:22.938840 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:22.938840 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:22.939930 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:22.939930 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:22.939930 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:22.939930 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:22.939930 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:22.939930 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:22.939930 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:22.939930 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:22.939930 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:22.939930 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:22.939930 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:22.939930 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:22.939930 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:22.939930 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:22.939930 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:22.939930 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:22.940859 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:22.940859 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:22.940859 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:22.940859 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:22.940859 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:22.940859 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:22.940859 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:22.940859 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:22.940859 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:22.940859 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:22.940859 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:23:22.940859 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:23:22.943375 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 12/27] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 18 16:23:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:23:22.944437 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c11de1-ebb3-46e1-b7dc-13b0cb6427ac None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:22.949151 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:23.037645 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 16/28] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:23.039111 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e4c8ba-5bbe-4917-b200-516e900484a9 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:23.104982 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c11de1-ebb3-46e1-b7dc-13b0cb6427ac None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:23.178195 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cbca6e-ae59-405d-8eba-7a48714070fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:23.179004 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cbca6e-ae59-405d-8eba-7a48714070fc None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:23.179292 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cbca6e-ae59-405d-8eba-7a48714070fc None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:23.179594 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cbca6e-ae59-405d-8eba-7a48714070fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:23.213119 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cbca6e-ae59-405d-8eba-7a48714070fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:23.214625 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cbca6e-ae59-405d-8eba-7a48714070fc None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:23.243844 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-e7cbca6e-ae59-405d-8eba-7a48714070fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:23:23.243844 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:23.243844 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:23.243844 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:23.243844 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:23.243844 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:23.243844 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:23.243844 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:23.243844 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:23.243844 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:23.243844 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:23.243844 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:23.243844 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:23:23.243844 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:23:23.243844 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:23:23.244711 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:23:23.244711 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:23.244711 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:23.244711 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:23.244711 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:23.244711 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:23.244711 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:23.244711 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:23.244711 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:23.244711 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:23.244711 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:23.244711 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:23.244711 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:23.244711 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:23.244711 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:23.244711 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:23.244711 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:23.245500 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:23:23.245500 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:23:23.245500 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:23:23.245500 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:23:23.245500 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 13/29] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:23:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:23:23.246274 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cbca6e-ae59-405d-8eba-7a48714070fc None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:23.254532 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e4c8ba-5bbe-4917-b200-516e900484a9 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:23.268599 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:23.313171 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fda7766-bde9-44f2-af4e-adb06ae91b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:23.313788 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda7766-bde9-44f2-af4e-adb06ae91b62 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:23.314311 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda7766-bde9-44f2-af4e-adb06ae91b62 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:23.315513 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda7766-bde9-44f2-af4e-adb06ae91b62 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:23.317108 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fda7766-bde9-44f2-af4e-adb06ae91b62 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:23.317722 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fda7766-bde9-44f2-af4e-adb06ae91b62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:23.320431 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fda7766-bde9-44f2-af4e-adb06ae91b62 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:23.345269 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 17/30] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:23:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:23.347731 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fda7766-bde9-44f2-af4e-adb06ae91b62 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:23.371749 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:23.431267 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cbca6e-ae59-405d-8eba-7a48714070fc None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:23.469459 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84ce7be-494e-458a-82f9-09e02e80d122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:23.470117 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ce7be-494e-458a-82f9-09e02e80d122 None admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:23.470410 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ce7be-494e-458a-82f9-09e02e80d122 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:23.470687 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ce7be-494e-458a-82f9-09e02e80d122 None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:23.471747 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84ce7be-494e-458a-82f9-09e02e80d122 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:23.473921 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84ce7be-494e-458a-82f9-09e02e80d122 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:23.483075 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fda7766-bde9-44f2-af4e-adb06ae91b62 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:23.521595 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2ec8cf-1c98-4c52-a110-ace3b4c02389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:23.522268 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ec8cf-1c98-4c52-a110-ace3b4c02389 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:23.522553 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ec8cf-1c98-4c52-a110-ace3b4c02389 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:23.522828 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ec8cf-1c98-4c52-a110-ace3b4c02389 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:23.523683 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2ec8cf-1c98-4c52-a110-ace3b4c02389 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:23.524466 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2ec8cf-1c98-4c52-a110-ace3b4c02389 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:23.525849 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2ec8cf-1c98-4c52-a110-ace3b4c02389 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:23.538098 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 18/31] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:23:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:23.543086 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2ec8cf-1c98-4c52-a110-ace3b4c02389 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:23.584463 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:23.659032 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2ec8cf-1c98-4c52-a110-ace3b4c02389 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:23.684521 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 14/32] 10.222.0.26 () {60 vars in 1131 bytes} [Mon Jul 18 16:23:23 2022] POST /identity/v3/roles => generated 219 bytes in 439 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:23:23.686271 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db39a2-e0e4-410e-9648-c1b7d3365de8 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:23.686665 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db39a2-e0e4-410e-9648-c1b7d3365de8 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:23.687075 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db39a2-e0e4-410e-9648-c1b7d3365de8 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:23.688598 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 15/33] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:23:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:23.689445 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db39a2-e0e4-410e-9648-c1b7d3365de8 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:23.730373 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66612a8-382b-4a3b-af49-fb7edf3efb58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:23.731283 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66612a8-382b-4a3b-af49-fb7edf3efb58 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:23.731596 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66612a8-382b-4a3b-af49-fb7edf3efb58 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:23.731876 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66612a8-382b-4a3b-af49-fb7edf3efb58 None admin] PATH_INFO: `/v3/users/e18df28d16ba462694258c7ad963170a` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:23.734634 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66612a8-382b-4a3b-af49-fb7edf3efb58 None admin] RBAC: Authorizing `identity:get_user(user_id=e18df28d16ba462694258c7ad963170a)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:23.736197 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66612a8-382b-4a3b-af49-fb7edf3efb58 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:23.738782 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 19/34] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:23 2022] GET /identity/v3/users/e18df28d16ba462694258c7ad963170a => generated 247 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:23.741631 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf841025-a480-43fc-be1d-701871ee4398 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:23.743396 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf841025-a480-43fc-be1d-701871ee4398 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:23.743690 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf841025-a480-43fc-be1d-701871ee4398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:23.782836 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-bf841025-a480-43fc-be1d-701871ee4398 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:23.785775 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-bf841025-a480-43fc-be1d-701871ee4398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:23.902897 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db39a2-e0e4-410e-9648-c1b7d3365de8 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:23.919269 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 20/35] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:23.919966 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf841025-a480-43fc-be1d-701871ee4398 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:23.923536 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:23.944935 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e780f24-5258-4358-bf73-6c48523e29b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:23.945672 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e780f24-5258-4358-bf73-6c48523e29b8 None admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:23.945970 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e780f24-5258-4358-bf73-6c48523e29b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:23.946248 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e780f24-5258-4358-bf73-6c48523e29b8 None admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:23.947099 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e780f24-5258-4358-bf73-6c48523e29b8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:23.950016 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:23.951325 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e780f24-5258-4358-bf73-6c48523e29b8 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:24.029839 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf841025-a480-43fc-be1d-701871ee4398 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:24.051517 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 16/36] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jul 18 16:23:23 2022] POST /identity/v3/projects => generated 301 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:23:24.051517 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59a2ad4-f1fb-4ccd-a766-a902f9aa168b None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:24.051517 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59a2ad4-f1fb-4ccd-a766-a902f9aa168b None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:24.051517 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59a2ad4-f1fb-4ccd-a766-a902f9aa168b None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:24.052200 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 17/37] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:23:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:24.055530 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59a2ad4-f1fb-4ccd-a766-a902f9aa168b None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:24.082962 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c455932-87d5-429f-95a6-8b8deaad0db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:24.084511 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c455932-87d5-429f-95a6-8b8deaad0db5 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:24.084810 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c455932-87d5-429f-95a6-8b8deaad0db5 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:24.085092 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c455932-87d5-429f-95a6-8b8deaad0db5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:24.087456 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c455932-87d5-429f-95a6-8b8deaad0db5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:24.089239 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c455932-87d5-429f-95a6-8b8deaad0db5 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:24.092040 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 21/38] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:23:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:24.093742 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58990efe-1f2e-46e2-87ff-4ad8b2f70dfb None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:24.094349 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58990efe-1f2e-46e2-87ff-4ad8b2f70dfb None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:24.094632 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58990efe-1f2e-46e2-87ff-4ad8b2f70dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:24.116306 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-58990efe-1f2e-46e2-87ff-4ad8b2f70dfb None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:24.117584 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-58990efe-1f2e-46e2-87ff-4ad8b2f70dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:24.183016 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59a2ad4-f1fb-4ccd-a766-a902f9aa168b None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:24.198022 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 22/39] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:24.199200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69769983-68cf-4ad9-8350-eb82f89ff3fc None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:24.199566 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69769983-68cf-4ad9-8350-eb82f89ff3fc None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:24.199801 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69769983-68cf-4ad9-8350-eb82f89ff3fc None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:24.201311 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 23/40] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:23:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:24.201843 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69769983-68cf-4ad9-8350-eb82f89ff3fc None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:24.243482 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:24.331144 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69769983-68cf-4ad9-8350-eb82f89ff3fc None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:24.343163 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5335d6-a94e-412a-9df5-fb9333e6ac54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:24.343873 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5335d6-a94e-412a-9df5-fb9333e6ac54 None admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:24.344161 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5335d6-a94e-412a-9df5-fb9333e6ac54 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:24.344467 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5335d6-a94e-412a-9df5-fb9333e6ac54 None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:24.345194 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5335d6-a94e-412a-9df5-fb9333e6ac54 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:24.346590 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5335d6-a94e-412a-9df5-fb9333e6ac54 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:24.365572 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c0a4aa-4c2e-4509-9e0b-d4503e946e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:24.366440 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0a4aa-4c2e-4509-9e0b-d4503e946e9e None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:24.366726 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0a4aa-4c2e-4509-9e0b-d4503e946e9e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:24.367007 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0a4aa-4c2e-4509-9e0b-d4503e946e9e None admin] PATH_INFO: `/v3/domains/default/users/e18df28d16ba462694258c7ad963170a/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:24.373484 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c0a4aa-4c2e-4509-9e0b-d4503e946e9e None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=e18df28d16ba462694258c7ad963170a, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:24.374938 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c0a4aa-4c2e-4509-9e0b-d4503e946e9e None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:24.406285 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 18/41] 10.222.0.26 () {60 vars in 1131 bytes} [Mon Jul 18 16:23:24 2022] POST /identity/v3/roles => generated 225 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:23:24.407590 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4773ffb-201c-4a26-8783-3da0eb9d5020 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:24.407884 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4773ffb-201c-4a26-8783-3da0eb9d5020 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:24.408166 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4773ffb-201c-4a26-8783-3da0eb9d5020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:24.427972 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-e4773ffb-201c-4a26-8783-3da0eb9d5020 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:24.429589 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-e4773ffb-201c-4a26-8783-3da0eb9d5020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:24.457638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:24.512712 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 19/42] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:24.513695 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4773ffb-201c-4a26-8783-3da0eb9d5020 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:24.549519 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 24/43] 10.222.0.26 () {58 vars in 1361 bytes} [Mon Jul 18 16:23:24 2022] PUT /identity/v3/domains/default/users/e18df28d16ba462694258c7ad963170a/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:23:24.551354 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c0a4aa-4c2e-4509-9e0b-d4503e946e9e None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:24.669699 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4773ffb-201c-4a26-8783-3da0eb9d5020 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:24.683482 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:24.703409 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c0a4aa-4c2e-4509-9e0b-d4503e946e9e None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:24.708729 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993efd7d-5031-4abd-982e-ac81d1ac1211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:24.709472 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993efd7d-5031-4abd-982e-ac81d1ac1211 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:24.709767 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993efd7d-5031-4abd-982e-ac81d1ac1211 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:24.710050 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993efd7d-5031-4abd-982e-ac81d1ac1211 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:24.711940 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993efd7d-5031-4abd-982e-ac81d1ac1211 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:24.713614 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993efd7d-5031-4abd-982e-ac81d1ac1211 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:24.715607 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 20/44] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:23:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:24.721310 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993efd7d-5031-4abd-982e-ac81d1ac1211 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:24.745312 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb8b56-052e-45f0-9880-dd1c1e91b77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:24.745990 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb8b56-052e-45f0-9880-dd1c1e91b77a None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:24.746274 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb8b56-052e-45f0-9880-dd1c1e91b77a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:24.746553 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb8b56-052e-45f0-9880-dd1c1e91b77a None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:24.748027 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcb8b56-052e-45f0-9880-dd1c1e91b77a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:24.749668 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcb8b56-052e-45f0-9880-dd1c1e91b77a None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:24.757603 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:24.836713 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993efd7d-5031-4abd-982e-ac81d1ac1211 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:24.849297 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 25/45] 10.222.0.26 () {60 vars in 1131 bytes} [Mon Jul 18 16:23:24 2022] POST /identity/v3/roles => generated 223 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:23:24.876802 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43519d9e-498e-4194-bb00-e44ff0a75572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:24.877503 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43519d9e-498e-4194-bb00-e44ff0a75572 None admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:24.877812 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43519d9e-498e-4194-bb00-e44ff0a75572 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:24.878215 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43519d9e-498e-4194-bb00-e44ff0a75572 None admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:24.878971 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43519d9e-498e-4194-bb00-e44ff0a75572 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:24.880546 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43519d9e-498e-4194-bb00-e44ff0a75572 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:24.944110 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 21/46] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jul 18 16:23:24 2022] POST /identity/v3/projects => generated 312 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:23:26.632669 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af6df0-9b27-488b-b93f-8c02dad99390 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:26.633717 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af6df0-9b27-488b-b93f-8c02dad99390 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:26.633853 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af6df0-9b27-488b-b93f-8c02dad99390 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:26.635265 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 26/47] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:23:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:26.640653 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc328d7-5077-40e1-ac61-dac5b46ca0b0 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:26.641085 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc328d7-5077-40e1-ac61-dac5b46ca0b0 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:26.641234 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc328d7-5077-40e1-ac61-dac5b46ca0b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:26.658890 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-0dc328d7-5077-40e1-ac61-dac5b46ca0b0 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:26.660143 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-0dc328d7-5077-40e1-ac61-dac5b46ca0b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:26.729877 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 22/48] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:26.752853 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8af6df0-9b27-488b-b93f-8c02dad99390 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:26.857653 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8af6df0-9b27-488b-b93f-8c02dad99390 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:26.890410 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e9e244-ef08-4572-a9e3-5560cd7c7c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:26.891124 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e9e244-ef08-4572-a9e3-5560cd7c7c4a None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:26.891407 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e9e244-ef08-4572-a9e3-5560cd7c7c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:26.891709 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e9e244-ef08-4572-a9e3-5560cd7c7c4a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:26.904745 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e9e244-ef08-4572-a9e3-5560cd7c7c4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:26.913489 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-46e9e244-ef08-4572-a9e3-5560cd7c7c4a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:23:26.913489 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:26.913489 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:26.913489 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:26.913489 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:26.913489 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:26.913489 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:26.913489 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:26.913489 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:26.913489 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:26.913489 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:26.913489 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:26.913489 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:26.913489 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:26.913489 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:26.913489 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:26.914674 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:26.914674 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:26.914674 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:26.914674 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:26.914674 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:26.914674 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:26.914674 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:26.914674 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:26.914674 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:26.914674 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:26.914674 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:26.914674 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:26.914674 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:26.914674 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:26.914674 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:26.914674 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:26.915426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:26.915426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:26.915426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:26.915426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:26.915426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:26.915426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:26.915426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:26.915426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:26.915426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:26.915426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:26.915426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:23:26.915426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:23:26.915426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 27/49] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 18 16:23:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:23:26.921320 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc328d7-5077-40e1-ac61-dac5b46ca0b0 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:26.931571 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:27.024081 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc328d7-5077-40e1-ac61-dac5b46ca0b0 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:27.040123 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd9204-5ea6-4f6d-90c7-9dda9ad6b663 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:27.040404 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd9204-5ea6-4f6d-90c7-9dda9ad6b663 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:27.040683 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd9204-5ea6-4f6d-90c7-9dda9ad6b663 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:27.042087 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 28/50] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:23:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:23:27.047157 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ede8eb-f73c-4ae1-8368-cff8fffa7466 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:27.047444 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ede8eb-f73c-4ae1-8368-cff8fffa7466 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:27.047771 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ede8eb-f73c-4ae1-8368-cff8fffa7466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:27.053691 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:27.059671 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a67546-7153-40bb-8d3f-14e35d3cdcfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:27.060385 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a67546-7153-40bb-8d3f-14e35d3cdcfc None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:27.060673 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a67546-7153-40bb-8d3f-14e35d3cdcfc None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:27.060953 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a67546-7153-40bb-8d3f-14e35d3cdcfc None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:27.061733 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a67546-7153-40bb-8d3f-14e35d3cdcfc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:27.062042 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a67546-7153-40bb-8d3f-14e35d3cdcfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:27.063535 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a67546-7153-40bb-8d3f-14e35d3cdcfc None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:27.074054 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 23/51] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:23:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:27.079978 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a67546-7153-40bb-8d3f-14e35d3cdcfc None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:27.154867 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:27.174119 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-f0ede8eb-f73c-4ae1-8368-cff8fffa7466 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:27.175693 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f0ede8eb-f73c-4ae1-8368-cff8fffa7466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:27.188141 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a67546-7153-40bb-8d3f-14e35d3cdcfc None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:27.222403 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66248b1-72b1-4c76-b963-fb5e2f60daf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:27.223288 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66248b1-72b1-4c76-b963-fb5e2f60daf9 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:27.223598 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66248b1-72b1-4c76-b963-fb5e2f60daf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:27.223880 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66248b1-72b1-4c76-b963-fb5e2f60daf9 None admin] PATH_INFO: `/v3/users/e18df28d16ba462694258c7ad963170a` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:27.225113 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66248b1-72b1-4c76-b963-fb5e2f60daf9 None admin] RBAC: Authorizing `identity:get_user(user_id=e18df28d16ba462694258c7ad963170a)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:27.226608 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66248b1-72b1-4c76-b963-fb5e2f60daf9 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:27.229195 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 24/52] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:27 2022] GET /identity/v3/users/e18df28d16ba462694258c7ad963170a => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:23:27.234573 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66248b1-72b1-4c76-b963-fb5e2f60daf9 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:27.245348 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 29/53] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:23:27.267961 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ede8eb-f73c-4ae1-8368-cff8fffa7466 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:27.335615 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66248b1-72b1-4c76-b963-fb5e2f60daf9 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:27.377207 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ede8eb-f73c-4ae1-8368-cff8fffa7466 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:27.377482 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54bf297-fcd4-4029-abe7-74b66d724f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:27.378298 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54bf297-fcd4-4029-abe7-74b66d724f5c None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:27.378591 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54bf297-fcd4-4029-abe7-74b66d724f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:27.378884 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54bf297-fcd4-4029-abe7-74b66d724f5c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:27.380137 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54bf297-fcd4-4029-abe7-74b66d724f5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:27.381629 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54bf297-fcd4-4029-abe7-74b66d724f5c None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:27.383403 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 25/54] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:23:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:23:27.389193 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54bf297-fcd4-4029-abe7-74b66d724f5c None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:27.398750 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:27.412812 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abdc642-13dc-4a4e-812c-46c888063ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:27.413545 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abdc642-13dc-4a4e-812c-46c888063ba9 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:27.413838 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abdc642-13dc-4a4e-812c-46c888063ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:27.414955 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abdc642-13dc-4a4e-812c-46c888063ba9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:27.416249 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abdc642-13dc-4a4e-812c-46c888063ba9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:27.417716 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abdc642-13dc-4a4e-812c-46c888063ba9 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:27.419370 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 30/55] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:23:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:27.425090 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abdc642-13dc-4a4e-812c-46c888063ba9 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:27.427978 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:27.492074 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54bf297-fcd4-4029-abe7-74b66d724f5c None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:27.527255 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efce0cd-ef19-446e-8446-79cdaa65496a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:27.527947 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efce0cd-ef19-446e-8446-79cdaa65496a None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:27.528076 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abdc642-13dc-4a4e-812c-46c888063ba9 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:27.528230 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efce0cd-ef19-446e-8446-79cdaa65496a None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:27.530785 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efce0cd-ef19-446e-8446-79cdaa65496a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:27.530785 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efce0cd-ef19-446e-8446-79cdaa65496a None admin] RBAC: Adding query filter params (role.id=98eaadb3669f4b608d572c001742f559, scope.domain.id=default, user.id=e18df28d16ba462694258c7ad963170a) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:27.530785 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efce0cd-ef19-446e-8446-79cdaa65496a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:27.531095 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efce0cd-ef19-446e-8446-79cdaa65496a None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:27.537388 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 26/56] 10.222.0.26 () {56 vars in 1322 bytes} [Mon Jul 18 16:23:27 2022] GET /identity/v3/role_assignments?user.id=e18df28d16ba462694258c7ad963170a&scope.domain.id=default&role.id=98eaadb3669f4b608d572c001742f559 => generated 530 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:23:27.566312 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e312e217-34c3-40cb-8055-b3b56cca188a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:27.567004 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312e217-34c3-40cb-8055-b3b56cca188a None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:27.567292 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312e217-34c3-40cb-8055-b3b56cca188a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:27.567609 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312e217-34c3-40cb-8055-b3b56cca188a None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:27.568352 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e312e217-34c3-40cb-8055-b3b56cca188a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:27.569861 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e312e217-34c3-40cb-8055-b3b56cca188a None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:27.681427 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 31/57] 10.222.0.26 () {60 vars in 1140 bytes} [Mon Jul 18 16:23:27 2022] POST /identity/v3/projects => generated 298 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:23:29.744722 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c061fd03-b5b3-4fd6-bd65-79bc94e932b5 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:29.745338 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c061fd03-b5b3-4fd6-bd65-79bc94e932b5 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:29.745338 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c061fd03-b5b3-4fd6-bd65-79bc94e932b5 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:29.746676 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 27/58] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:23:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:29.752211 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad9e853-3934-40e1-a93d-05e8d152ac08 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:29.752637 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad9e853-3934-40e1-a93d-05e8d152ac08 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:29.752770 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad9e853-3934-40e1-a93d-05e8d152ac08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:29.770598 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-aad9e853-3934-40e1-a93d-05e8d152ac08 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:29.771820 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-aad9e853-3934-40e1-a93d-05e8d152ac08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:29.827086 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:29.846021 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 32/59] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:29.871136 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c061fd03-b5b3-4fd6-bd65-79bc94e932b5 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:29.942540 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:29.975105 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c061fd03-b5b3-4fd6-bd65-79bc94e932b5 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:30.010538 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2f78d6-156f-4e98-a9b9-a073b31db9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:30.011251 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f78d6-156f-4e98-a9b9-a073b31db9b7 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:30.011565 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f78d6-156f-4e98-a9b9-a073b31db9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:30.011850 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f78d6-156f-4e98-a9b9-a073b31db9b7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:30.013092 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2f78d6-156f-4e98-a9b9-a073b31db9b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:30.014569 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2f78d6-156f-4e98-a9b9-a073b31db9b7 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:30.016250 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 28/60] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:23:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:30.021900 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad9e853-3934-40e1-a93d-05e8d152ac08 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:30.122728 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad9e853-3934-40e1-a93d-05e8d152ac08 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:30.156926 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a56c49-6359-4665-81e0-e583f9c3c1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:30.157592 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a56c49-6359-4665-81e0-e583f9c3c1c1 None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:30.157878 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a56c49-6359-4665-81e0-e583f9c3c1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:30.158157 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a56c49-6359-4665-81e0-e583f9c3c1c1 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:30.158894 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a56c49-6359-4665-81e0-e583f9c3c1c1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:30.160386 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a56c49-6359-4665-81e0-e583f9c3c1c1 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:30.271399 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 33/61] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 18 16:23:30 2022] POST /identity/v3/users => generated 275 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:23:33.552382 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c062ba7-adf6-4930-8f82-ebba72d68208 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:33.553244 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c062ba7-adf6-4930-8f82-ebba72d68208 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:33.553662 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c062ba7-adf6-4930-8f82-ebba72d68208 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:33.555752 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 29/62] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:23:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:33.561727 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d728e-8c13-435f-a6ec-b44d7cc0a52d None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:33.562521 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d728e-8c13-435f-a6ec-b44d7cc0a52d None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:33.562939 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d728e-8c13-435f-a6ec-b44d7cc0a52d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:33.587480 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-ae2d728e-8c13-435f-a6ec-b44d7cc0a52d None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:33.588989 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-ae2d728e-8c13-435f-a6ec-b44d7cc0a52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:33.637698 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:33.673636 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 34/63] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:33.716989 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c062ba7-adf6-4930-8f82-ebba72d68208 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:33.743238 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232cf52-4eb0-4e90-a2a2-26f1d6530459 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:33.744278 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232cf52-4eb0-4e90-a2a2-26f1d6530459 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:33.744529 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232cf52-4eb0-4e90-a2a2-26f1d6530459 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:33.745945 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 35/64] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:23:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:33.752653 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ea1e61-fc86-4b02-b741-84cad200c822 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:33.752922 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ea1e61-fc86-4b02-b741-84cad200c822 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:33.753200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ea1e61-fc86-4b02-b741-84cad200c822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:33.771018 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-38ea1e61-fc86-4b02-b741-84cad200c822 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:33.772554 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-38ea1e61-fc86-4b02-b741-84cad200c822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:33.851516 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 36/65] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:23:33.876619 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ea1e61-fc86-4b02-b741-84cad200c822 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:33.926228 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c062ba7-adf6-4930-8f82-ebba72d68208 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:33.967675 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc740bbe-6f3a-4612-81be-e9a67c1fdba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:33.968506 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc740bbe-6f3a-4612-81be-e9a67c1fdba2 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:33.968799 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc740bbe-6f3a-4612-81be-e9a67c1fdba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:33.969082 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc740bbe-6f3a-4612-81be-e9a67c1fdba2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:33.976743 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:33.981895 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc740bbe-6f3a-4612-81be-e9a67c1fdba2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:33.988984 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:33.995618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-cc740bbe-6f3a-4612-81be-e9a67c1fdba2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:23:33.995618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:33.995618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:33.995618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:33.995618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:33.995618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:33.995618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:33.995618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:33.995618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:33.995618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:33.995618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:33.995618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:33.995618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:33.995618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:33.995618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:33.995618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:33.996472 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:33.996472 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:33.996472 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:33.996472 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:33.996472 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:33.996472 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:33.996472 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:33.996472 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:33.996472 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:33.996472 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:33.996472 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:33.996472 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:33.996472 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:33.996472 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:33.996472 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:33.996472 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:33.997222 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:33.997222 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:33.997222 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:33.997222 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:33.997222 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:33.997222 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:33.997222 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:33.997222 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:33.997222 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:33.997222 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:33.997222 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:23:33.997222 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:23:33.997222 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 30/66] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 18 16:23:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:23:33.997827 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ea1e61-fc86-4b02-b741-84cad200c822 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:33.997899 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f408c91-cb41-48ea-a39e-4f2fe139c367 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:33.997899 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f408c91-cb41-48ea-a39e-4f2fe139c367 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:33.997899 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f408c91-cb41-48ea-a39e-4f2fe139c367 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:33.997899 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 31/67] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:23:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:33.997899 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d267d-9e8c-4f51-9736-595b0ad406e2 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:33.997899 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d267d-9e8c-4f51-9736-595b0ad406e2 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:33.998205 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d267d-9e8c-4f51-9736-595b0ad406e2 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:34.003371 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 32/68] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:23:33 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:34.005015 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9421a7f-94a2-4df4-a5c1-81536132cbc3 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:34.005311 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9421a7f-94a2-4df4-a5c1-81536132cbc3 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:34.005596 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9421a7f-94a2-4df4-a5c1-81536132cbc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:34.025134 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-f9421a7f-94a2-4df4-a5c1-81536132cbc3 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:34.026346 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-f9421a7f-94a2-4df4-a5c1-81536132cbc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:34.035369 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a9c540-0066-40bf-8354-0f57ae82d812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:34.036308 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9c540-0066-40bf-8354-0f57ae82d812 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:34.036669 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9c540-0066-40bf-8354-0f57ae82d812 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:34.036965 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9c540-0066-40bf-8354-0f57ae82d812 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:34.049596 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a9c540-0066-40bf-8354-0f57ae82d812 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:34.058631 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-f6a9c540-0066-40bf-8354-0f57ae82d812 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:23:34.058631 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:34.058631 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:34.058631 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:34.058631 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:34.058631 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:34.058631 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:34.058631 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:34.058631 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:34.058631 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:34.058631 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:34.058631 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:34.058631 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:34.058631 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:34.058631 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:34.058631 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:34.059692 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:34.059692 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:34.059692 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:34.059692 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:34.059692 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:34.059692 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:34.059692 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:34.059692 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:34.059692 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:34.059692 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:34.059692 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:34.059692 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:34.059692 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:34.059692 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:34.059692 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:34.059692 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:34.060552 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:34.060552 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:34.060552 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:34.060552 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:34.060552 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:34.060552 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:34.060552 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:34.060552 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:34.060552 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:34.060552 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:34.060552 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:23:34.060552 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:23:34.062803 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 37/69] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:23:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:23:34.063925 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a9c540-0066-40bf-8354-0f57ae82d812 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:34.106613 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 33/70] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:34.108291 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba818cc-f414-40cc-8176-59cba9de78a9 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:34.108587 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba818cc-f414-40cc-8176-59cba9de78a9 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:34.108869 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba818cc-f414-40cc-8176-59cba9de78a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:34.129494 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-bba818cc-f414-40cc-8176-59cba9de78a9 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:34.130972 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-bba818cc-f414-40cc-8176-59cba9de78a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:34.186423 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a9c540-0066-40bf-8354-0f57ae82d812 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:34.205833 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 34/71] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:34.206907 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba818cc-f414-40cc-8176-59cba9de78a9 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:34.231185 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165a4504-1d99-4803-b5fb-f92fed7e480a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:34.231980 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a4504-1d99-4803-b5fb-f92fed7e480a None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:34.232275 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a4504-1d99-4803-b5fb-f92fed7e480a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:34.232561 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a4504-1d99-4803-b5fb-f92fed7e480a None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:34.233336 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165a4504-1d99-4803-b5fb-f92fed7e480a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:34.233651 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165a4504-1d99-4803-b5fb-f92fed7e480a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:34.235078 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165a4504-1d99-4803-b5fb-f92fed7e480a None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:34.244296 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 38/72] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:23:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:34.245244 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165a4504-1d99-4803-b5fb-f92fed7e480a None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:34.272029 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:34.285529 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:34.330692 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba818cc-f414-40cc-8176-59cba9de78a9 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:34.353569 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165a4504-1d99-4803-b5fb-f92fed7e480a None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:34.371156 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bad4bb-a2f5-4d5e-81f1-2cdafed1efc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:34.371795 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bad4bb-a2f5-4d5e-81f1-2cdafed1efc6 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:34.372084 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bad4bb-a2f5-4d5e-81f1-2cdafed1efc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:34.372368 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bad4bb-a2f5-4d5e-81f1-2cdafed1efc6 None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:34.373134 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bad4bb-a2f5-4d5e-81f1-2cdafed1efc6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:34.373448 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bad4bb-a2f5-4d5e-81f1-2cdafed1efc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:34.374862 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bad4bb-a2f5-4d5e-81f1-2cdafed1efc6 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:34.383382 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 35/73] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 18 16:23:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:34.384264 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bad4bb-a2f5-4d5e-81f1-2cdafed1efc6 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:34.395417 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b6dfd9-8878-400b-a3af-e36520f5e3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:34.396276 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b6dfd9-8878-400b-a3af-e36520f5e3fb None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:34.396568 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b6dfd9-8878-400b-a3af-e36520f5e3fb None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:34.396850 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b6dfd9-8878-400b-a3af-e36520f5e3fb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:34.408208 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b6dfd9-8878-400b-a3af-e36520f5e3fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:34.417445 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-20b6dfd9-8878-400b-a3af-e36520f5e3fb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:23:34.417445 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:34.417445 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:34.417445 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:34.417445 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:34.417445 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:34.417445 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:34.417445 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:34.417445 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:34.417445 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:34.417445 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:34.417445 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:34.417445 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:34.417445 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:34.417445 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:34.417445 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:34.418445 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:34.418445 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:34.418445 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:34.418445 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:34.418445 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:34.418445 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:34.418445 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:34.418445 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:34.418445 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:34.418445 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:34.418445 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:34.418445 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:34.418445 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:34.418445 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:34.418445 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:34.418445 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:34.419373 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:34.419373 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:34.419373 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:34.419373 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:34.419373 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:34.419373 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:34.419373 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:34.419373 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:34.419373 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:34.419373 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:34.419373 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:23:34.419373 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:23:34.419373 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 39/74] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 18 16:23:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:23:34.420168 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b6dfd9-8878-400b-a3af-e36520f5e3fb None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:34.499542 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bad4bb-a2f5-4d5e-81f1-2cdafed1efc6 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:34.509338 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:34.519391 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:34.528155 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b6dfd9-8878-400b-a3af-e36520f5e3fb None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:34.533318 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d8a3f-f57f-4bb8-9d32-e2c8a1dc99f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:34.534117 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d8a3f-f57f-4bb8-9d32-e2c8a1dc99f3 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:34.534398 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d8a3f-f57f-4bb8-9d32-e2c8a1dc99f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:34.534673 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d8a3f-f57f-4bb8-9d32-e2c8a1dc99f3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:34.545567 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17d8a3f-f57f-4bb8-9d32-e2c8a1dc99f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:34.553274 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-a17d8a3f-f57f-4bb8-9d32-e2c8a1dc99f3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:23:34.553274 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:34.553274 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:34.553274 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:34.553274 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:34.553274 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:34.553274 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:34.553274 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:34.553274 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:34.553274 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:34.553274 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:34.553274 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:34.553274 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:34.553274 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:34.553274 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:34.553274 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:34.554591 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:34.554591 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:34.554591 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:34.554591 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:34.554591 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:34.554591 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:34.554591 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:34.554591 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:34.554591 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:34.554591 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:34.554591 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:34.554591 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:34.554591 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:34.554591 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:34.554591 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:34.554591 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:34.556540 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:34.556540 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:34.556540 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:34.556540 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:34.556540 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:34.556540 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:34.556540 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:34.556540 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:34.556540 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:34.556540 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:34.556540 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:23:34.556540 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:23:34.556540 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 36/75] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:23:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:23:34.556540 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d8a3f-f57f-4bb8-9d32-e2c8a1dc99f3 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:34.571318 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c048f44d-b8ef-409a-b516-6f8d9ff054f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:34.572863 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048f44d-b8ef-409a-b516-6f8d9ff054f5 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:34.573150 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048f44d-b8ef-409a-b516-6f8d9ff054f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:34.573427 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048f44d-b8ef-409a-b516-6f8d9ff054f5 None admin] PATH_INFO: `/v3/users/e18df28d16ba462694258c7ad963170a` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:34.574648 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c048f44d-b8ef-409a-b516-6f8d9ff054f5 None admin] RBAC: Authorizing `identity:get_user(user_id=e18df28d16ba462694258c7ad963170a)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:34.576170 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c048f44d-b8ef-409a-b516-6f8d9ff054f5 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:34.577963 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 40/76] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:34 2022] GET /identity/v3/users/e18df28d16ba462694258c7ad963170a => generated 247 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:34.578566 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c048f44d-b8ef-409a-b516-6f8d9ff054f5 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:34.658117 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d8a3f-f57f-4bb8-9d32-e2c8a1dc99f3 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:34.688318 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c048f44d-b8ef-409a-b516-6f8d9ff054f5 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:34.692441 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49809511-2300-483e-b833-e7c0fa46afc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:34.693389 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49809511-2300-483e-b833-e7c0fa46afc7 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:34.693706 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49809511-2300-483e-b833-e7c0fa46afc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:34.693989 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49809511-2300-483e-b833-e7c0fa46afc7 None admin] PATH_INFO: `/v3/users/cda57e656e834b9792465e79b138c9c4` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:34.707822 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49809511-2300-483e-b833-e7c0fa46afc7 None admin] RBAC: Authorizing `identity:get_user(user_id=cda57e656e834b9792465e79b138c9c4)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:34.709340 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49809511-2300-483e-b833-e7c0fa46afc7 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:34.711083 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 37/77] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:34 2022] GET /identity/v3/users/cda57e656e834b9792465e79b138c9c4 => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:34.712591 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f8b4fd-81d8-4d70-9801-6da291f5ffd5 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:34.712890 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f8b4fd-81d8-4d70-9801-6da291f5ffd5 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:34.713182 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f8b4fd-81d8-4d70-9801-6da291f5ffd5 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:34.714574 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 38/78] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:23:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:34.715292 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f8b4fd-81d8-4d70-9801-6da291f5ffd5 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:34.726660 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8205edd8-ee99-44de-b090-22320d94ec4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:34.727958 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8205edd8-ee99-44de-b090-22320d94ec4f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:34.728244 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8205edd8-ee99-44de-b090-22320d94ec4f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:34.728517 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8205edd8-ee99-44de-b090-22320d94ec4f None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:34.729270 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8205edd8-ee99-44de-b090-22320d94ec4f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:34.729576 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8205edd8-ee99-44de-b090-22320d94ec4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:34.730965 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8205edd8-ee99-44de-b090-22320d94ec4f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:34.739131 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 41/79] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:23:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:34.739969 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8205edd8-ee99-44de-b090-22320d94ec4f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:34.759203 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:34.761001 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:34.826542 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f8b4fd-81d8-4d70-9801-6da291f5ffd5 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:34.849494 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8205edd8-ee99-44de-b090-22320d94ec4f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:34.861624 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096699a2-3782-41e9-a5f7-d058a5a25618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:34.862306 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096699a2-3782-41e9-a5f7-d058a5a25618 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:34.862598 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096699a2-3782-41e9-a5f7-d058a5a25618 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:34.862883 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096699a2-3782-41e9-a5f7-d058a5a25618 None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:34.863686 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096699a2-3782-41e9-a5f7-d058a5a25618 None admin] RBAC: Adding query filter params (name=member) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:34.863999 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096699a2-3782-41e9-a5f7-d058a5a25618 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:34.865378 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096699a2-3782-41e9-a5f7-d058a5a25618 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:34.874491 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 39/80] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 18 16:23:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:34.875304 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096699a2-3782-41e9-a5f7-d058a5a25618 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:34.889744 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b8c0bd-308c-4608-9f28-8d94a3c2f2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:34.890411 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b8c0bd-308c-4608-9f28-8d94a3c2f2fb None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:34.890696 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b8c0bd-308c-4608-9f28-8d94a3c2f2fb None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:34.890978 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b8c0bd-308c-4608-9f28-8d94a3c2f2fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:34.966492 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b8c0bd-308c-4608-9f28-8d94a3c2f2fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:34.967966 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b8c0bd-308c-4608-9f28-8d94a3c2f2fb None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:34.977757 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:34.981394 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096699a2-3782-41e9-a5f7-d058a5a25618 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:34.983210 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:35.017902 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de753154-e066-41b9-8e92-ea1dcab324ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:35.018569 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de753154-e066-41b9-8e92-ea1dcab324ec None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:35.018860 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de753154-e066-41b9-8e92-ea1dcab324ec None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:35.019144 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de753154-e066-41b9-8e92-ea1dcab324ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:35.071717 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 42/81] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:23:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:23:35.072992 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638fc30-728d-46eb-a367-dbdef8b12ff2 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:35.073287 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638fc30-728d-46eb-a367-dbdef8b12ff2 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:35.073568 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638fc30-728d-46eb-a367-dbdef8b12ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:35.092050 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de753154-e066-41b9-8e92-ea1dcab324ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:35.092330 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-3638fc30-728d-46eb-a367-dbdef8b12ff2 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:35.093534 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-3638fc30-728d-46eb-a367-dbdef8b12ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:35.093627 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de753154-e066-41b9-8e92-ea1dcab324ec None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:35.163197 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 43/82] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:35.164101 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3638fc30-728d-46eb-a367-dbdef8b12ff2 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:35.192623 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:35.196660 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 40/83] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:23:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:23:35.197478 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de753154-e066-41b9-8e92-ea1dcab324ec None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:35.223396 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:35.281697 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3638fc30-728d-46eb-a367-dbdef8b12ff2 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:35.312526 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de753154-e066-41b9-8e92-ea1dcab324ec None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:35.316887 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bbb80b-1dd5-46dc-883e-016bccf33afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:35.317773 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bbb80b-1dd5-46dc-883e-016bccf33afb None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:35.318055 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bbb80b-1dd5-46dc-883e-016bccf33afb None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:35.318333 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bbb80b-1dd5-46dc-883e-016bccf33afb None admin] PATH_INFO: `/v3/users/cda57e656e834b9792465e79b138c9c4` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:35.319560 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bbb80b-1dd5-46dc-883e-016bccf33afb None admin] RBAC: Authorizing `identity:get_user(user_id=cda57e656e834b9792465e79b138c9c4)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:35.321123 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bbb80b-1dd5-46dc-883e-016bccf33afb None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:35.323112 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 44/84] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:35 2022] GET /identity/v3/users/cda57e656e834b9792465e79b138c9c4 => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:35.323740 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bbb80b-1dd5-46dc-883e-016bccf33afb None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:35.355303 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b15357-12f5-4247-aef1-8444e54de154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:35.356225 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b15357-12f5-4247-aef1-8444e54de154 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:35.356511 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b15357-12f5-4247-aef1-8444e54de154 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:35.356786 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b15357-12f5-4247-aef1-8444e54de154 None admin] PATH_INFO: `/v3/users/cda57e656e834b9792465e79b138c9c4` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:35.358017 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b15357-12f5-4247-aef1-8444e54de154 None admin] RBAC: Authorizing `identity:get_user(user_id=cda57e656e834b9792465e79b138c9c4)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:35.359508 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b15357-12f5-4247-aef1-8444e54de154 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:35.361186 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 41/85] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:35 2022] GET /identity/v3/users/cda57e656e834b9792465e79b138c9c4 => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:35.361944 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b15357-12f5-4247-aef1-8444e54de154 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:35.429181 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:35.433193 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bbb80b-1dd5-46dc-883e-016bccf33afb None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:35.443879 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:35.465657 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b15357-12f5-4247-aef1-8444e54de154 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:35.467881 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49a581-d7fe-428f-8f15-e2614a2fba53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:35.468753 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49a581-d7fe-428f-8f15-e2614a2fba53 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:35.469041 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49a581-d7fe-428f-8f15-e2614a2fba53 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:35.469324 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49a581-d7fe-428f-8f15-e2614a2fba53 None admin] PATH_INFO: `/v3/projects/04afdb2deaea49078a14a588c9d6de7e` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:35.470609 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf49a581-d7fe-428f-8f15-e2614a2fba53 None admin] RBAC: Authorizing `identity:get_project(project_id=04afdb2deaea49078a14a588c9d6de7e)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:35.472111 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf49a581-d7fe-428f-8f15-e2614a2fba53 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:35.473904 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 45/86] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:23:35 2022] GET /identity/v3/projects/04afdb2deaea49078a14a588c9d6de7e => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:35.474627 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49a581-d7fe-428f-8f15-e2614a2fba53 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:35.500091 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0387e22-dbdd-4170-ad6e-f2d7c4fabe7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:35.500898 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0387e22-dbdd-4170-ad6e-f2d7c4fabe7d None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:35.501187 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0387e22-dbdd-4170-ad6e-f2d7c4fabe7d None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:35.501473 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0387e22-dbdd-4170-ad6e-f2d7c4fabe7d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:35.502684 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0387e22-dbdd-4170-ad6e-f2d7c4fabe7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:35.504190 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0387e22-dbdd-4170-ad6e-f2d7c4fabe7d None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:35.505829 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 42/87] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:23:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:35.506596 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0387e22-dbdd-4170-ad6e-f2d7c4fabe7d None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:35.576128 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49a581-d7fe-428f-8f15-e2614a2fba53 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:35.607681 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0387e22-dbdd-4170-ad6e-f2d7c4fabe7d None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:35.609957 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce147b4a-8053-4664-8582-64e45a730749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:35.610817 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce147b4a-8053-4664-8582-64e45a730749 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:35.611097 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce147b4a-8053-4664-8582-64e45a730749 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:35.611389 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce147b4a-8053-4664-8582-64e45a730749 None admin] PATH_INFO: `/v3/projects/04afdb2deaea49078a14a588c9d6de7e` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:35.612598 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce147b4a-8053-4664-8582-64e45a730749 None admin] RBAC: Authorizing `identity:get_project(project_id=04afdb2deaea49078a14a588c9d6de7e)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:35.614097 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce147b4a-8053-4664-8582-64e45a730749 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:35.615844 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 46/88] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:23:35 2022] GET /identity/v3/projects/04afdb2deaea49078a14a588c9d6de7e => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:35.616647 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce147b4a-8053-4664-8582-64e45a730749 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:35.642146 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43929f6c-b42b-40ad-a22b-dd5380e281bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:35.642796 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43929f6c-b42b-40ad-a22b-dd5380e281bf None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:35.643078 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43929f6c-b42b-40ad-a22b-dd5380e281bf None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:35.643361 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43929f6c-b42b-40ad-a22b-dd5380e281bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:35.647205 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:35.663796 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:35.717327 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43929f6c-b42b-40ad-a22b-dd5380e281bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:35.718262 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce147b4a-8053-4664-8582-64e45a730749 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:35.718798 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43929f6c-b42b-40ad-a22b-dd5380e281bf None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:35.753990 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313f07e6-ae00-4b6f-ad7d-5805d3ad79c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:35.754645 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f07e6-ae00-4b6f-ad7d-5805d3ad79c0 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:35.754932 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f07e6-ae00-4b6f-ad7d-5805d3ad79c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:35.755218 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f07e6-ae00-4b6f-ad7d-5805d3ad79c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:35.820208 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 43/89] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:23:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:23:35.821072 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43929f6c-b42b-40ad-a22b-dd5380e281bf None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:35.823353 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313f07e6-ae00-4b6f-ad7d-5805d3ad79c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:35.825048 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313f07e6-ae00-4b6f-ad7d-5805d3ad79c0 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:35.858251 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:35.871998 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:35.924900 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43929f6c-b42b-40ad-a22b-dd5380e281bf None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:35.927928 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 47/90] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:23:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:23:35.928797 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313f07e6-ae00-4b6f-ad7d-5805d3ad79c0 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:35.969409 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd916ae-da2e-426f-9334-976d5b854138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:35.970076 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd916ae-da2e-426f-9334-976d5b854138 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:35.970374 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd916ae-da2e-426f-9334-976d5b854138 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:35.970687 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd916ae-da2e-426f-9334-976d5b854138 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:35.971563 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd916ae-da2e-426f-9334-976d5b854138 None admin] RBAC: Adding query filter params (role.id=98eaadb3669f4b608d572c001742f559, scope.project.id=04afdb2deaea49078a14a588c9d6de7e, user.id=e18df28d16ba462694258c7ad963170a) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:35.971887 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd916ae-da2e-426f-9334-976d5b854138 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:35.973337 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd916ae-da2e-426f-9334-976d5b854138 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:35.979838 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 44/91] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 18 16:23:35 2022] GET /identity/v3/role_assignments?user.id=e18df28d16ba462694258c7ad963170a&scope.project.id=04afdb2deaea49078a14a588c9d6de7e&role.id=98eaadb3669f4b608d572c001742f559 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:35.980645 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd916ae-da2e-426f-9334-976d5b854138 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:36.041494 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313f07e6-ae00-4b6f-ad7d-5805d3ad79c0 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:36.082064 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586ea918-edf2-42ca-aa38-25f1324f35a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:36.083136 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ea918-edf2-42ca-aa38-25f1324f35a3 None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:36.083691 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ea918-edf2-42ca-aa38-25f1324f35a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:36.084168 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ea918-edf2-42ca-aa38-25f1324f35a3 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:36.085082 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586ea918-edf2-42ca-aa38-25f1324f35a3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:36.086794 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586ea918-edf2-42ca-aa38-25f1324f35a3 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:36.105964 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:36.109328 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd916ae-da2e-426f-9334-976d5b854138 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:36.145710 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7961e4f1-e58e-4451-9830-d1959f1926a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:36.146375 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961e4f1-e58e-4451-9830-d1959f1926a6 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:36.146659 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961e4f1-e58e-4451-9830-d1959f1926a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:36.146941 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961e4f1-e58e-4451-9830-d1959f1926a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:36.147840 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7961e4f1-e58e-4451-9830-d1959f1926a6 None admin] RBAC: Adding query filter params (role.id=aa0552e8ba924f5d9cb1740da7d4a4dc, scope.project.id=04afdb2deaea49078a14a588c9d6de7e, user.id=cda57e656e834b9792465e79b138c9c4) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:36.148149 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7961e4f1-e58e-4451-9830-d1959f1926a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:36.149831 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7961e4f1-e58e-4451-9830-d1959f1926a6 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:36.155838 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 45/92] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 18 16:23:35 2022] GET /identity/v3/role_assignments?user.id=cda57e656e834b9792465e79b138c9c4&scope.project.id=04afdb2deaea49078a14a588c9d6de7e&role.id=aa0552e8ba924f5d9cb1740da7d4a4dc => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:36.156653 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7961e4f1-e58e-4451-9830-d1959f1926a6 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:36.170670 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 48/93] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 18 16:23:35 2022] POST /identity/v3/users => generated 289 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:23:36.182202 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586ea918-edf2-42ca-aa38-25f1324f35a3 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:36.191521 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:36.371716 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7961e4f1-e58e-4451-9830-d1959f1926a6 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:36.405459 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586ea918-edf2-42ca-aa38-25f1324f35a3 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:36.414321 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4abc38-b9f0-4af2-8d40-2f7190df71a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:36.417059 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4abc38-b9f0-4af2-8d40-2f7190df71a1 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:36.417493 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4abc38-b9f0-4af2-8d40-2f7190df71a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:36.417911 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4abc38-b9f0-4af2-8d40-2f7190df71a1 None admin] PATH_INFO: `/v3/projects/04afdb2deaea49078a14a588c9d6de7e` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:36.419434 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4abc38-b9f0-4af2-8d40-2f7190df71a1 None admin] RBAC: Authorizing `identity:get_project(project_id=04afdb2deaea49078a14a588c9d6de7e)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:36.420941 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4abc38-b9f0-4af2-8d40-2f7190df71a1 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:36.422874 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 46/94] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:23:36 2022] GET /identity/v3/projects/04afdb2deaea49078a14a588c9d6de7e => generated 298 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:36.429046 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4abc38-b9f0-4af2-8d40-2f7190df71a1 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:36.470539 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899e11f6-18ff-4265-b504-6fff0ddc5245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:36.471412 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e11f6-18ff-4265-b504-6fff0ddc5245 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:36.471729 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e11f6-18ff-4265-b504-6fff0ddc5245 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:36.472008 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e11f6-18ff-4265-b504-6fff0ddc5245 None admin] PATH_INFO: `/v3/projects/277b079204de49d98ea603246c8a67c6` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:36.473859 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899e11f6-18ff-4265-b504-6fff0ddc5245 None admin] RBAC: Authorizing `identity:get_project(project_id=277b079204de49d98ea603246c8a67c6)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:36.475304 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899e11f6-18ff-4265-b504-6fff0ddc5245 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:36.477607 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 49/95] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:23:36 2022] GET /identity/v3/projects/277b079204de49d98ea603246c8a67c6 => generated 312 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:36.483166 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:36.486253 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899e11f6-18ff-4265-b504-6fff0ddc5245 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:36.577910 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:36.581133 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4abc38-b9f0-4af2-8d40-2f7190df71a1 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:36.628025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899e11f6-18ff-4265-b504-6fff0ddc5245 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:36.631154 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d5512a-6ec7-4a39-8e06-37a594a901ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:36.632454 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5512a-6ec7-4a39-8e06-37a594a901ed None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:36.632947 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5512a-6ec7-4a39-8e06-37a594a901ed None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:36.634093 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5512a-6ec7-4a39-8e06-37a594a901ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:36.635164 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d5512a-6ec7-4a39-8e06-37a594a901ed None admin] RBAC: Adding query filter params (role.id=729cd9c0f42d47b6a8eee01e5f5a2d28, scope.project.id=04afdb2deaea49078a14a588c9d6de7e, user.id=cda57e656e834b9792465e79b138c9c4) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:36.635744 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d5512a-6ec7-4a39-8e06-37a594a901ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:36.637450 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d5512a-6ec7-4a39-8e06-37a594a901ed None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:36.644142 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 47/96] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 18 16:23:36 2022] GET /identity/v3/role_assignments?user.id=cda57e656e834b9792465e79b138c9c4&scope.project.id=04afdb2deaea49078a14a588c9d6de7e&role.id=729cd9c0f42d47b6a8eee01e5f5a2d28 => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:36.701620 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d097d2bc-9de4-40b8-995f-4bbf3c477cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:36.702303 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097d2bc-9de4-40b8-995f-4bbf3c477cb7 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:36.702592 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097d2bc-9de4-40b8-995f-4bbf3c477cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:36.702872 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097d2bc-9de4-40b8-995f-4bbf3c477cb7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:36.703950 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d097d2bc-9de4-40b8-995f-4bbf3c477cb7 None admin] RBAC: Adding query filter params (role.id=aa0552e8ba924f5d9cb1740da7d4a4dc, scope.project.id=277b079204de49d98ea603246c8a67c6, user.id=cda57e656e834b9792465e79b138c9c4) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:36.704236 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d097d2bc-9de4-40b8-995f-4bbf3c477cb7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:36.705623 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d097d2bc-9de4-40b8-995f-4bbf3c477cb7 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:36.721258 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 50/97] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 18 16:23:36 2022] GET /identity/v3/role_assignments?user.id=cda57e656e834b9792465e79b138c9c4&scope.project.id=277b079204de49d98ea603246c8a67c6&role.id=aa0552e8ba924f5d9cb1740da7d4a4dc => generated 259 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:38.673150 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5bdf51-fe7d-4052-bb5b-a50e77977d1c None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:38.673764 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5bdf51-fe7d-4052-bb5b-a50e77977d1c None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:38.673764 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5bdf51-fe7d-4052-bb5b-a50e77977d1c None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:38.675271 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 48/98] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:23:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:38.686704 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e94f2a-0fe6-4f09-9026-cf63437ffbfd None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:38.687226 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e94f2a-0fe6-4f09-9026-cf63437ffbfd None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:38.687307 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e94f2a-0fe6-4f09-9026-cf63437ffbfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:38.709459 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-a9e94f2a-0fe6-4f09-9026-cf63437ffbfd None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:38.710998 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a9e94f2a-0fe6-4f09-9026-cf63437ffbfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:38.793634 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 51/99] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:38.834297 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5bdf51-fe7d-4052-bb5b-a50e77977d1c None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:38.837000 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2607c01-cf40-4e6b-b71a-cc1048bd86ef None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:38.837522 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2607c01-cf40-4e6b-b71a-cc1048bd86ef None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:38.838085 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2607c01-cf40-4e6b-b71a-cc1048bd86ef None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:38.839737 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 52/100] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:23:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:23:38.845689 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d66572-bc5b-4f89-abd3-cbb7502575a1 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:38.846293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d66572-bc5b-4f89-abd3-cbb7502575a1 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:38.846783 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d66572-bc5b-4f89-abd3-cbb7502575a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:38.876367 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-97d66572-bc5b-4f89-abd3-cbb7502575a1 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:38.878030 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-97d66572-bc5b-4f89-abd3-cbb7502575a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:39.015073 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:39.017602 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:39.021323 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5bdf51-fe7d-4052-bb5b-a50e77977d1c None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:39.042162 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 53/101] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:23:39.074007 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d66572-bc5b-4f89-abd3-cbb7502575a1 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:39.104841 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c998620d-d790-4c05-ac74-1438584db258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:39.105676 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998620d-d790-4c05-ac74-1438584db258 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:39.105960 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998620d-d790-4c05-ac74-1438584db258 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:39.106237 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998620d-d790-4c05-ac74-1438584db258 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:39.140459 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c998620d-d790-4c05-ac74-1438584db258 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:39.148539 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-c998620d-d790-4c05-ac74-1438584db258 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:23:39.148539 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:39.148539 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:39.148539 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:39.148539 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:39.148539 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:39.148539 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:39.148539 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:39.148539 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:39.148539 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:39.148539 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:39.148539 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:39.148539 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:39.148539 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:39.148539 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:39.148539 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:39.149539 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:39.149539 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:39.149539 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:39.149539 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:39.149539 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:39.149539 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:39.149539 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:39.149539 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:39.149539 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:39.149539 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:39.149539 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:39.149539 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:39.149539 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:39.149539 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:39.149539 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:39.149539 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:39.151251 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:39.151251 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:39.151251 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:39.151251 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:39.151251 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:39.151251 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:39.151251 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:39.151251 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:39.151251 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:39.151251 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:39.151251 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:23:39.151251 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:23:39.155911 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 49/102] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 18 16:23:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:23:39.162337 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c998620d-d790-4c05-ac74-1438584db258 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:39.295555 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d66572-bc5b-4f89-abd3-cbb7502575a1 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:39.359570 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf82b4f-a5cb-425f-b73a-ee2b4d74b04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:39.360933 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf82b4f-a5cb-425f-b73a-ee2b4d74b04a None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:39.361397 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf82b4f-a5cb-425f-b73a-ee2b4d74b04a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:39.361842 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf82b4f-a5cb-425f-b73a-ee2b4d74b04a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:39.373938 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf82b4f-a5cb-425f-b73a-ee2b4d74b04a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:39.382730 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-9cf82b4f-a5cb-425f-b73a-ee2b4d74b04a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:23:39.382730 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:39.382730 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:39.382730 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:39.382730 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:39.382730 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:39.382730 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:39.382730 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:39.382730 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:39.382730 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:39.382730 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:39.382730 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:39.382730 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:39.382730 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:39.382730 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:39.382730 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:39.384358 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:39.384358 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:39.384358 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:39.384358 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:39.384358 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:39.384358 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:39.384358 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:39.384358 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:39.384358 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:39.384358 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:39.384358 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:39.384358 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:39.384358 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:39.384358 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:39.384358 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:39.384358 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:39.385207 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:39.385207 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:39.385207 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:39.385207 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:39.385207 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:39.385207 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:39.385207 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:39.385207 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:39.385207 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:39.385207 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:39.385207 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:23:39.385207 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:23:39.387456 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 54/103] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:23:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:23:39.389025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d8c5f1-1e44-4e23-abe2-4b5367156ace None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:39.389549 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d8c5f1-1e44-4e23-abe2-4b5367156ace None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:39.389982 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d8c5f1-1e44-4e23-abe2-4b5367156ace None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:39.391449 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 55/104] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:23:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:39.393226 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d8c5f1-1e44-4e23-abe2-4b5367156ace None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:39.395552 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c998620d-d790-4c05-ac74-1438584db258 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:39.439790 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb6d53a-d82c-481f-aa2b-bec121317b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:39.440477 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6d53a-d82c-481f-aa2b-bec121317b27 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:39.440770 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6d53a-d82c-481f-aa2b-bec121317b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:39.441055 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6d53a-d82c-481f-aa2b-bec121317b27 None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:39.441821 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb6d53a-d82c-481f-aa2b-bec121317b27 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:39.442134 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb6d53a-d82c-481f-aa2b-bec121317b27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:39.443547 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb6d53a-d82c-481f-aa2b-bec121317b27 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:39.452695 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 50/105] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:23:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:39.454112 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29990a1a-b7f0-407e-bc2f-4ded6203efda None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:39.454417 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29990a1a-b7f0-407e-bc2f-4ded6203efda None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:39.454705 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29990a1a-b7f0-407e-bc2f-4ded6203efda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:39.471991 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-29990a1a-b7f0-407e-bc2f-4ded6203efda None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:39.473202 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-29990a1a-b7f0-407e-bc2f-4ded6203efda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:39.474501 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:39.491908 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:39.530837 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d8c5f1-1e44-4e23-abe2-4b5367156ace None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:39.547783 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 51/106] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:39.548976 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efff98b-9808-4e91-82c7-9e8d738482e3 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:39.549275 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efff98b-9808-4e91-82c7-9e8d738482e3 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:39.549554 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efff98b-9808-4e91-82c7-9e8d738482e3 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:39.554991 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 52/107] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:23:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:39.556613 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efff98b-9808-4e91-82c7-9e8d738482e3 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:39.574374 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f53364-03ae-40cc-9b7d-cd6b35fcd048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:39.576394 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f53364-03ae-40cc-9b7d-cd6b35fcd048 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:39.576394 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f53364-03ae-40cc-9b7d-cd6b35fcd048 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:39.576394 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f53364-03ae-40cc-9b7d-cd6b35fcd048 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:39.576629 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f53364-03ae-40cc-9b7d-cd6b35fcd048 None admin] RBAC: Adding query filter params (name=member) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:39.576797 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f53364-03ae-40cc-9b7d-cd6b35fcd048 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:39.578378 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f53364-03ae-40cc-9b7d-cd6b35fcd048 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:39.588786 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 56/108] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 18 16:23:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:39.590138 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b57ad6-00dd-4150-acbc-62a4c7265445 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:39.590468 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b57ad6-00dd-4150-acbc-62a4c7265445 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:39.590749 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b57ad6-00dd-4150-acbc-62a4c7265445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:39.705113 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-48b57ad6-00dd-4150-acbc-62a4c7265445 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:39.706340 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-48b57ad6-00dd-4150-acbc-62a4c7265445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:39.778196 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 57/109] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:39.781014 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efff98b-9808-4e91-82c7-9e8d738482e3 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:39.784178 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b57ad6-00dd-4150-acbc-62a4c7265445 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:39.825339 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cda8b43-e700-4836-8573-bbad4792c12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:39.826468 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda8b43-e700-4836-8573-bbad4792c12b None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:39.827594 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda8b43-e700-4836-8573-bbad4792c12b None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:39.827930 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda8b43-e700-4836-8573-bbad4792c12b None admin] PATH_INFO: `/v3/users/e18df28d16ba462694258c7ad963170a` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:39.831321 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cda8b43-e700-4836-8573-bbad4792c12b None admin] RBAC: Authorizing `identity:get_user(user_id=e18df28d16ba462694258c7ad963170a)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:39.833626 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cda8b43-e700-4836-8573-bbad4792c12b None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:39.835412 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 53/110] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:39 2022] GET /identity/v3/users/e18df28d16ba462694258c7ad963170a => generated 247 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:39.836211 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cda8b43-e700-4836-8573-bbad4792c12b None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:39.861914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:39.863013 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:39.925707 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b57ad6-00dd-4150-acbc-62a4c7265445 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:39.964975 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d78eaeb-54e3-4d86-9e5d-91e3ab0e8832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:39.966015 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78eaeb-54e3-4d86-9e5d-91e3ab0e8832 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:39.966330 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78eaeb-54e3-4d86-9e5d-91e3ab0e8832 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:39.966714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78eaeb-54e3-4d86-9e5d-91e3ab0e8832 None admin] PATH_INFO: `/v3/users/cda57e656e834b9792465e79b138c9c4` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:39.967998 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d78eaeb-54e3-4d86-9e5d-91e3ab0e8832 None admin] RBAC: Authorizing `identity:get_user(user_id=cda57e656e834b9792465e79b138c9c4)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:39.969467 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d78eaeb-54e3-4d86-9e5d-91e3ab0e8832 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:39.969665 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cda8b43-e700-4836-8573-bbad4792c12b None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:39.971276 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 58/111] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:39 2022] GET /identity/v3/users/cda57e656e834b9792465e79b138c9c4 => generated 275 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:39.972275 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d78eaeb-54e3-4d86-9e5d-91e3ab0e8832 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:40.011790 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa494634-2d3d-4452-bb8d-e70977252359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:40.012607 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa494634-2d3d-4452-bb8d-e70977252359 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:40.012892 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa494634-2d3d-4452-bb8d-e70977252359 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:40.013167 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa494634-2d3d-4452-bb8d-e70977252359 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:40.023771 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa494634-2d3d-4452-bb8d-e70977252359 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:40.032459 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-fa494634-2d3d-4452-bb8d-e70977252359 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:23:40.032459 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:40.032459 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:40.032459 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:40.032459 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:40.032459 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:40.032459 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:40.032459 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:40.032459 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:40.032459 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:40.032459 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:40.032459 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:40.032459 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:40.032459 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:40.032459 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:40.032459 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:40.033696 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:40.033696 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:40.033696 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:40.033696 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:40.033696 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:40.033696 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:40.033696 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:40.033696 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:40.033696 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:40.033696 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:40.033696 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:40.033696 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:40.033696 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:40.033696 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:40.033696 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:40.033696 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:40.034706 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:40.034706 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:40.034706 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:40.034706 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:40.034706 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:40.034706 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:40.034706 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:40.034706 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:40.034706 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:40.034706 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:40.034706 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:23:40.034706 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:23:40.034706 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 54/112] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 18 16:23:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:23:40.036085 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa494634-2d3d-4452-bb8d-e70977252359 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:40.077837 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d78eaeb-54e3-4d86-9e5d-91e3ab0e8832 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:40.092600 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:40.112533 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56871940-fd4d-4320-a26b-0f10e8b7db0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:40.113003 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:40.113329 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56871940-fd4d-4320-a26b-0f10e8b7db0f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:40.113617 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56871940-fd4d-4320-a26b-0f10e8b7db0f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:40.113895 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56871940-fd4d-4320-a26b-0f10e8b7db0f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:40.115084 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56871940-fd4d-4320-a26b-0f10e8b7db0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:40.116677 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56871940-fd4d-4320-a26b-0f10e8b7db0f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:40.123355 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 59/113] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:23:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:40.124116 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56871940-fd4d-4320-a26b-0f10e8b7db0f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:40.137781 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa494634-2d3d-4452-bb8d-e70977252359 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:40.173022 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be637ab-acb1-44c7-a5ab-7561c4816c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:40.173903 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be637ab-acb1-44c7-a5ab-7561c4816c21 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:40.174196 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be637ab-acb1-44c7-a5ab-7561c4816c21 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:40.174487 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be637ab-acb1-44c7-a5ab-7561c4816c21 None admin] PATH_INFO: `/v3/projects/04afdb2deaea49078a14a588c9d6de7e` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:40.175800 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be637ab-acb1-44c7-a5ab-7561c4816c21 None admin] RBAC: Authorizing `identity:get_project(project_id=04afdb2deaea49078a14a588c9d6de7e)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:40.177268 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be637ab-acb1-44c7-a5ab-7561c4816c21 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:40.179002 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 55/114] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:23:40 2022] GET /identity/v3/projects/04afdb2deaea49078a14a588c9d6de7e => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:40.180325 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78b4a9-bafe-4243-b5ce-10c11f673a1f None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:40.180647 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78b4a9-bafe-4243-b5ce-10c11f673a1f None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:40.180647 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78b4a9-bafe-4243-b5ce-10c11f673a1f None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:40.182140 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 56/115] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:23:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:40.183226 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b9ac7-0632-4b45-84c4-00e76aaea3bb None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:40.183720 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b9ac7-0632-4b45-84c4-00e76aaea3bb None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:40.184121 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b9ac7-0632-4b45-84c4-00e76aaea3bb None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:40.185297 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 57/116] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:23:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:40.186100 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138b9ac7-0632-4b45-84c4-00e76aaea3bb None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:40.234120 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56871940-fd4d-4320-a26b-0f10e8b7db0f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:40.268761 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4b589c-c89a-42f6-938e-e9618e5e4c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:40.269623 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4b589c-c89a-42f6-938e-e9618e5e4c7f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:40.269914 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4b589c-c89a-42f6-938e-e9618e5e4c7f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:40.270194 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4b589c-c89a-42f6-938e-e9618e5e4c7f None admin] PATH_INFO: `/v3/projects/04afdb2deaea49078a14a588c9d6de7e` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:40.271393 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4b589c-c89a-42f6-938e-e9618e5e4c7f None admin] RBAC: Authorizing `identity:get_project(project_id=04afdb2deaea49078a14a588c9d6de7e)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:40.272870 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4b589c-c89a-42f6-938e-e9618e5e4c7f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:40.274770 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 60/117] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:23:40 2022] GET /identity/v3/projects/04afdb2deaea49078a14a588c9d6de7e => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:40.275535 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4b589c-c89a-42f6-938e-e9618e5e4c7f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:40.292946 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138b9ac7-0632-4b45-84c4-00e76aaea3bb None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:40.327969 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:40.329321 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e363c2d-96bd-4c43-8a3f-d782ba65c80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:40.330001 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e363c2d-96bd-4c43-8a3f-d782ba65c80e None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:40.330292 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e363c2d-96bd-4c43-8a3f-d782ba65c80e None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:40.330581 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e363c2d-96bd-4c43-8a3f-d782ba65c80e None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:40.331331 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e363c2d-96bd-4c43-8a3f-d782ba65c80e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:40.331679 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e363c2d-96bd-4c43-8a3f-d782ba65c80e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:40.333174 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e363c2d-96bd-4c43-8a3f-d782ba65c80e None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:40.341455 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 58/118] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:23:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:40.342315 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e363c2d-96bd-4c43-8a3f-d782ba65c80e None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:40.351826 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:40.378627 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4b589c-c89a-42f6-938e-e9618e5e4c7f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:40.412936 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9b7070-1188-4b85-accb-e9e240ccee92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:40.413603 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b7070-1188-4b85-accb-e9e240ccee92 None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:40.413893 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b7070-1188-4b85-accb-e9e240ccee92 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:40.414177 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b7070-1188-4b85-accb-e9e240ccee92 None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:40.415447 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9b7070-1188-4b85-accb-e9e240ccee92 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:40.416954 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9b7070-1188-4b85-accb-e9e240ccee92 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:40.441836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e363c2d-96bd-4c43-8a3f-d782ba65c80e None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:40.474583 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb30ec-57c9-4e6b-ba97-175e14bd422d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:40.475479 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb30ec-57c9-4e6b-ba97-175e14bd422d None admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:40.475817 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb30ec-57c9-4e6b-ba97-175e14bd422d None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:40.476094 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb30ec-57c9-4e6b-ba97-175e14bd422d None admin] PATH_INFO: `/v3/projects/04afdb2deaea49078a14a588c9d6de7e/users/e18df28d16ba462694258c7ad963170a/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:40.478209 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbb30ec-57c9-4e6b-ba97-175e14bd422d None admin] RBAC: Authorizing `identity:create_grant(project_id=04afdb2deaea49078a14a588c9d6de7e, user_id=e18df28d16ba462694258c7ad963170a, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:40.479692 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbb30ec-57c9-4e6b-ba97-175e14bd422d None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:40.512598 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 61/119] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jul 18 16:23:40 2022] POST /identity/v3/projects => generated 302 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:23:40.513869 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605a37d-63bf-45aa-bc88-6380fdc0b170 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:40.514161 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605a37d-63bf-45aa-bc88-6380fdc0b170 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:40.514440 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605a37d-63bf-45aa-bc88-6380fdc0b170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:40.532050 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-c605a37d-63bf-45aa-bc88-6380fdc0b170 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:40.538417 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-c605a37d-63bf-45aa-bc88-6380fdc0b170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:40.595682 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 59/120] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 18 16:23:40 2022] PUT /identity/v3/projects/04afdb2deaea49078a14a588c9d6de7e/users/e18df28d16ba462694258c7ad963170a/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:23:40.598269 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8454f6-b004-406a-96e1-c196087084f9 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:40.598964 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8454f6-b004-406a-96e1-c196087084f9 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:40.599523 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8454f6-b004-406a-96e1-c196087084f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:40.612682 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 62/121] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:40.614810 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c605a37d-63bf-45aa-bc88-6380fdc0b170 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:40.644948 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-0c8454f6-b004-406a-96e1-c196087084f9 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:40.646513 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-0c8454f6-b004-406a-96e1-c196087084f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:40.661014 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:40.737832 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:40.740551 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 60/122] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:40.744181 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8454f6-b004-406a-96e1-c196087084f9 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:40.754634 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c605a37d-63bf-45aa-bc88-6380fdc0b170 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:40.794212 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3165016-a869-4b86-bc01-7d5b2c9e9f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:40.795132 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3165016-a869-4b86-bc01-7d5b2c9e9f38 None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:40.795415 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3165016-a869-4b86-bc01-7d5b2c9e9f38 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:40.795725 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3165016-a869-4b86-bc01-7d5b2c9e9f38 None admin] PATH_INFO: `/v3/projects/04afdb2deaea49078a14a588c9d6de7e/users/cda57e656e834b9792465e79b138c9c4/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:40.797813 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3165016-a869-4b86-bc01-7d5b2c9e9f38 None admin] RBAC: Authorizing `identity:create_grant(project_id=04afdb2deaea49078a14a588c9d6de7e, user_id=cda57e656e834b9792465e79b138c9c4, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:40.799290 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3165016-a869-4b86-bc01-7d5b2c9e9f38 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:40.862721 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8454f6-b004-406a-96e1-c196087084f9 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:40.900617 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 63/123] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 18 16:23:40 2022] PUT /identity/v3/projects/04afdb2deaea49078a14a588c9d6de7e/users/cda57e656e834b9792465e79b138c9c4/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:23:40.901505 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3165016-a869-4b86-bc01-7d5b2c9e9f38 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:40.919705 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d112eb-84ca-49d8-bece-093f03da5ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:40.921162 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d112eb-84ca-49d8-bece-093f03da5ec6 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:40.921684 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d112eb-84ca-49d8-bece-093f03da5ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:40.922208 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d112eb-84ca-49d8-bece-093f03da5ec6 None admin] PATH_INFO: `/v3/users/cda57e656e834b9792465e79b138c9c4` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:40.926472 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d112eb-84ca-49d8-bece-093f03da5ec6 None admin] RBAC: Authorizing `identity:get_user(user_id=cda57e656e834b9792465e79b138c9c4)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:40.930514 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d112eb-84ca-49d8-bece-093f03da5ec6 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:40.935906 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 61/124] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:40 2022] GET /identity/v3/users/cda57e656e834b9792465e79b138c9c4 => generated 275 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:40.939343 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d112eb-84ca-49d8-bece-093f03da5ec6 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:41.060879 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:41.114617 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:41.124868 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3165016-a869-4b86-bc01-7d5b2c9e9f38 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:41.133335 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d112eb-84ca-49d8-bece-093f03da5ec6 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:41.161959 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90517c0-a314-4ce0-aea2-8b724aa90bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:41.162853 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90517c0-a314-4ce0-aea2-8b724aa90bbf None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:41.163206 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90517c0-a314-4ce0-aea2-8b724aa90bbf None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:41.163591 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90517c0-a314-4ce0-aea2-8b724aa90bbf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:41.178977 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988d0936-2a0a-45c8-b2ea-c800d095880c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:41.179816 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d0936-2a0a-45c8-b2ea-c800d095880c None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:41.180097 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d0936-2a0a-45c8-b2ea-c800d095880c None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:41.180370 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d0936-2a0a-45c8-b2ea-c800d095880c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:41.180766 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90517c0-a314-4ce0-aea2-8b724aa90bbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:41.192164 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988d0936-2a0a-45c8-b2ea-c800d095880c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:41.193294 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-f90517c0-a314-4ce0-aea2-8b724aa90bbf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:23:41.193294 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:41.193294 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:41.193294 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:41.193294 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:41.193294 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:41.193294 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:41.193294 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:41.193294 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:41.193294 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:41.193294 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:41.193294 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:41.193294 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:41.193294 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:41.193294 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:41.193294 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:41.194130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:41.194130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:41.194130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:41.194130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:41.194130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:41.194130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:41.194130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:41.194130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:41.194130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:41.194130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:41.194130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:41.194130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:41.194130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:41.194130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:41.194130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:41.194130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:41.194891 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:41.194891 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:41.194891 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:41.194891 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:41.194891 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:41.194891 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:41.194891 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:41.194891 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:41.194891 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:41.194891 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:41.194891 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:23:41.194891 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:23:41.197461 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 64/125] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:23:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:23:41.198470 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90517c0-a314-4ce0-aea2-8b724aa90bbf None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:41.201023 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-988d0936-2a0a-45c8-b2ea-c800d095880c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:23:41.201023 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:41.201023 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:41.201023 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:41.201023 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:41.201023 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:41.201023 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:41.201023 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:41.201023 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:41.201023 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:41.201023 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:41.201023 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:41.201023 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:41.201023 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:41.201023 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:41.201023 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:41.201876 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:41.201876 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:41.201876 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:41.201876 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:41.201876 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:41.201876 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:41.201876 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:41.201876 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:41.201876 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:41.201876 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:41.201876 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:41.201876 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:41.201876 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:41.201876 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:41.201876 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:41.201876 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:41.202742 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:41.202742 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:41.202742 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:41.202742 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:41.202742 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:41.202742 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:41.202742 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:41.202742 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:41.202742 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:41.202742 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:41.202742 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:23:41.202742 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:23:41.202742 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 62/126] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:23:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:23:41.203359 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988d0936-2a0a-45c8-b2ea-c800d095880c None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:41.365897 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90517c0-a314-4ce0-aea2-8b724aa90bbf None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:41.421537 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988d0936-2a0a-45c8-b2ea-c800d095880c None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:41.426094 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38910324-fa1e-492d-8fab-5310a21ece8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:41.426979 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38910324-fa1e-492d-8fab-5310a21ece8e None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:41.427322 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38910324-fa1e-492d-8fab-5310a21ece8e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:41.427645 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38910324-fa1e-492d-8fab-5310a21ece8e None admin] PATH_INFO: `/v3/projects/04afdb2deaea49078a14a588c9d6de7e` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:41.428919 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38910324-fa1e-492d-8fab-5310a21ece8e None admin] RBAC: Authorizing `identity:get_project(project_id=04afdb2deaea49078a14a588c9d6de7e)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:41.431838 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38910324-fa1e-492d-8fab-5310a21ece8e None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:41.433648 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 65/127] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:23:41 2022] GET /identity/v3/projects/04afdb2deaea49078a14a588c9d6de7e => generated 298 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:41.434511 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38910324-fa1e-492d-8fab-5310a21ece8e None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:41.454802 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:41.469855 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7198299-22eb-47f3-9782-10b82a3d7926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:41.470593 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7198299-22eb-47f3-9782-10b82a3d7926 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:41.470881 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7198299-22eb-47f3-9782-10b82a3d7926 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:41.471163 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7198299-22eb-47f3-9782-10b82a3d7926 None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:41.471956 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7198299-22eb-47f3-9782-10b82a3d7926 None admin] RBAC: Adding query filter params (name=member) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:41.473677 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7198299-22eb-47f3-9782-10b82a3d7926 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:41.473768 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7198299-22eb-47f3-9782-10b82a3d7926 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:41.483875 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:41.493529 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 63/128] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 18 16:23:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:41.494364 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7198299-22eb-47f3-9782-10b82a3d7926 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:41.646658 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7198299-22eb-47f3-9782-10b82a3d7926 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:41.669724 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38910324-fa1e-492d-8fab-5310a21ece8e None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:41.688159 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4831043-4491-4222-a59c-6265da0e4096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:41.689066 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4831043-4491-4222-a59c-6265da0e4096 None admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:41.689865 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4831043-4491-4222-a59c-6265da0e4096 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:41.690155 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4831043-4491-4222-a59c-6265da0e4096 None admin] PATH_INFO: `/v3/projects/04afdb2deaea49078a14a588c9d6de7e/users/cda57e656e834b9792465e79b138c9c4/roles/729cd9c0f42d47b6a8eee01e5f5a2d28` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:41.693287 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4831043-4491-4222-a59c-6265da0e4096 None admin] RBAC: Authorizing `identity:create_grant(project_id=04afdb2deaea49078a14a588c9d6de7e, user_id=cda57e656e834b9792465e79b138c9c4, role_id=729cd9c0f42d47b6a8eee01e5f5a2d28)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:41.694768 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4831043-4491-4222-a59c-6265da0e4096 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:41.725067 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892881c5-ed79-45fa-a077-3f290e1a705c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:41.725754 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892881c5-ed79-45fa-a077-3f290e1a705c None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:41.726098 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892881c5-ed79-45fa-a077-3f290e1a705c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:41.726383 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892881c5-ed79-45fa-a077-3f290e1a705c None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:41.727837 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892881c5-ed79-45fa-a077-3f290e1a705c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:41.728363 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892881c5-ed79-45fa-a077-3f290e1a705c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:41.729761 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892881c5-ed79-45fa-a077-3f290e1a705c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:41.738905 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 66/129] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 18 16:23:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:41.739880 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892881c5-ed79-45fa-a077-3f290e1a705c None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:41.749330 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 64/130] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 18 16:23:41 2022] PUT /identity/v3/projects/04afdb2deaea49078a14a588c9d6de7e/users/cda57e656e834b9792465e79b138c9c4/roles/729cd9c0f42d47b6a8eee01e5f5a2d28 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:23:41.750223 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4831043-4491-4222-a59c-6265da0e4096 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:41.848961 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:41.943575 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:41.964652 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4831043-4491-4222-a59c-6265da0e4096 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:41.969856 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892881c5-ed79-45fa-a077-3f290e1a705c None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:42.028684 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b76009a-e2f1-4aa5-8646-ef6083098761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:42.030831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebd1c0f-a131-4228-a6e9-67551acb04e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:42.031952 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd1c0f-a131-4228-a6e9-67551acb04e7 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:42.032375 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b76009a-e2f1-4aa5-8646-ef6083098761 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:42.032809 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b76009a-e2f1-4aa5-8646-ef6083098761 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:42.033249 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b76009a-e2f1-4aa5-8646-ef6083098761 None admin] PATH_INFO: `/v3/users/77e423c38dcf4d16a61998f6eb9e4ca3` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:42.033703 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd1c0f-a131-4228-a6e9-67551acb04e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:42.034806 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd1c0f-a131-4228-a6e9-67551acb04e7 None admin] PATH_INFO: `/v3/users/cda57e656e834b9792465e79b138c9c4` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:42.036415 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebd1c0f-a131-4228-a6e9-67551acb04e7 None admin] RBAC: Authorizing `identity:get_user(user_id=cda57e656e834b9792465e79b138c9c4)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:42.039389 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebd1c0f-a131-4228-a6e9-67551acb04e7 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:42.043357 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 67/131] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:41 2022] GET /identity/v3/users/cda57e656e834b9792465e79b138c9c4 => generated 275 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:42.049014 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebd1c0f-a131-4228-a6e9-67551acb04e7 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:42.070097 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b76009a-e2f1-4aa5-8646-ef6083098761 None admin] RBAC: Authorizing `identity:get_user(user_id=77e423c38dcf4d16a61998f6eb9e4ca3)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:42.071980 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b76009a-e2f1-4aa5-8646-ef6083098761 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:42.073900 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 65/132] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:41 2022] GET /identity/v3/users/77e423c38dcf4d16a61998f6eb9e4ca3 => generated 289 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:42.084611 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b76009a-e2f1-4aa5-8646-ef6083098761 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:42.176936 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebd1c0f-a131-4228-a6e9-67551acb04e7 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:42.214290 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9525a8-66d2-45c5-8e74-b2a6f054deaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:42.215387 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9525a8-66d2-45c5-8e74-b2a6f054deaf None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:42.215844 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9525a8-66d2-45c5-8e74-b2a6f054deaf None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:42.216285 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9525a8-66d2-45c5-8e74-b2a6f054deaf None admin] PATH_INFO: `/v3/projects/277b079204de49d98ea603246c8a67c6` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:42.217681 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9525a8-66d2-45c5-8e74-b2a6f054deaf None admin] RBAC: Authorizing `identity:get_project(project_id=277b079204de49d98ea603246c8a67c6)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:42.219291 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9525a8-66d2-45c5-8e74-b2a6f054deaf None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:42.221302 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 68/133] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:23:42 2022] GET /identity/v3/projects/277b079204de49d98ea603246c8a67c6 => generated 312 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:23:42.227685 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9525a8-66d2-45c5-8e74-b2a6f054deaf None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:42.270902 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:42.323706 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b76009a-e2f1-4aa5-8646-ef6083098761 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:42.331889 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:42.423047 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ebf79-f2f7-4e6c-bd54-9cc4c74fa240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:42.423991 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ebf79-f2f7-4e6c-bd54-9cc4c74fa240 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:42.424427 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ebf79-f2f7-4e6c-bd54-9cc4c74fa240 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:42.424849 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ebf79-f2f7-4e6c-bd54-9cc4c74fa240 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:42.460757 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9525a8-66d2-45c5-8e74-b2a6f054deaf None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:42.503791 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1538d820-f578-4512-90fa-a92034faad0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:42.505401 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538d820-f578-4512-90fa-a92034faad0b None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:42.505847 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538d820-f578-4512-90fa-a92034faad0b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:42.506275 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538d820-f578-4512-90fa-a92034faad0b None admin] PATH_INFO: `/v3/projects/277b079204de49d98ea603246c8a67c6/users/cda57e656e834b9792465e79b138c9c4/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:42.508970 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1538d820-f578-4512-90fa-a92034faad0b None admin] RBAC: Authorizing `identity:create_grant(project_id=277b079204de49d98ea603246c8a67c6, user_id=cda57e656e834b9792465e79b138c9c4, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:42.510792 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1538d820-f578-4512-90fa-a92034faad0b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:42.558573 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94ebf79-f2f7-4e6c-bd54-9cc4c74fa240 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:42.560375 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94ebf79-f2f7-4e6c-bd54-9cc4c74fa240 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:42.587204 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 69/134] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 18 16:23:42 2022] PUT /identity/v3/projects/277b079204de49d98ea603246c8a67c6/users/cda57e656e834b9792465e79b138c9c4/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:23:42.683622 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 66/135] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:23:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:23:42.692738 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1538d820-f578-4512-90fa-a92034faad0b None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:42.785433 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:42.809262 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1538d820-f578-4512-90fa-a92034faad0b None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:42.853629 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c452a0d0-41ef-4ec9-8350-356a6d7e3570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:42.854557 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c452a0d0-41ef-4ec9-8350-356a6d7e3570 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:42.854851 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c452a0d0-41ef-4ec9-8350-356a6d7e3570 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:42.855134 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c452a0d0-41ef-4ec9-8350-356a6d7e3570 None admin] PATH_INFO: `/v3/projects/04afdb2deaea49078a14a588c9d6de7e` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:42.856394 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c452a0d0-41ef-4ec9-8350-356a6d7e3570 None admin] RBAC: Authorizing `identity:get_project(project_id=04afdb2deaea49078a14a588c9d6de7e)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:42.857914 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c452a0d0-41ef-4ec9-8350-356a6d7e3570 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:42.859845 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 70/136] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:23:42 2022] GET /identity/v3/projects/04afdb2deaea49078a14a588c9d6de7e => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:42.865915 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ebf79-f2f7-4e6c-bd54-9cc4c74fa240 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:42.869899 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:42.911461 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a5997-fcd0-4313-91c9-18675c07d77c None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:42.911808 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a5997-fcd0-4313-91c9-18675c07d77c None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:42.912095 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a5997-fcd0-4313-91c9-18675c07d77c None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:42.913497 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 71/137] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:23:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:42.919528 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2608821d-413b-4580-b690-3f3706595309 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:42.919850 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2608821d-413b-4580-b690-3f3706595309 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:42.920112 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2608821d-413b-4580-b690-3f3706595309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:42.950205 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-2608821d-413b-4580-b690-3f3706595309 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:42.955360 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-2608821d-413b-4580-b690-3f3706595309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:43.032042 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ebf79-f2f7-4e6c-bd54-9cc4c74fa240 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:43.099780 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9face743-9d13-4a24-a56b-0605a8c4d75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:43.101391 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 72/138] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:43.102951 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9face743-9d13-4a24-a56b-0605a8c4d75d None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:43.103650 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9face743-9d13-4a24-a56b-0605a8c4d75d None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:43.104155 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9face743-9d13-4a24-a56b-0605a8c4d75d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:43.105350 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9face743-9d13-4a24-a56b-0605a8c4d75d None admin] RBAC: Adding query filter params (role.id=2ba8d51abb6e4efa872c82ac8c9ba4a8, scope.project.id=04afdb2deaea49078a14a588c9d6de7e, user.id=77e423c38dcf4d16a61998f6eb9e4ca3) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:43.105863 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9face743-9d13-4a24-a56b-0605a8c4d75d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:43.107638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9face743-9d13-4a24-a56b-0605a8c4d75d None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:43.114524 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 67/139] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 18 16:23:42 2022] GET /identity/v3/role_assignments?user.id=77e423c38dcf4d16a61998f6eb9e4ca3&scope.project.id=04afdb2deaea49078a14a588c9d6de7e&role.id=2ba8d51abb6e4efa872c82ac8c9ba4a8 => generated 259 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:43.136200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2608821d-413b-4580-b690-3f3706595309 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:43.245643 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:43.279816 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2608821d-413b-4580-b690-3f3706595309 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:43.330440 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991b691e-8dc7-46a4-847a-486361369427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:43.331663 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b691e-8dc7-46a4-847a-486361369427 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:43.332157 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b691e-8dc7-46a4-847a-486361369427 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:43.332657 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b691e-8dc7-46a4-847a-486361369427 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:43.334099 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991b691e-8dc7-46a4-847a-486361369427 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:43.335826 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991b691e-8dc7-46a4-847a-486361369427 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:43.338329 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 73/140] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:23:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:43.344094 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9face743-9d13-4a24-a56b-0605a8c4d75d None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:43.437432 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:43.478692 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9face743-9d13-4a24-a56b-0605a8c4d75d None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:43.517579 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76332040-1505-4ead-8848-bd09c01b437c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:43.518338 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76332040-1505-4ead-8848-bd09c01b437c None admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:43.518620 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76332040-1505-4ead-8848-bd09c01b437c None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:43.518925 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76332040-1505-4ead-8848-bd09c01b437c None admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:43.519704 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76332040-1505-4ead-8848-bd09c01b437c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:43.521204 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76332040-1505-4ead-8848-bd09c01b437c None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:43.588893 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c1603-aa1f-45c3-adb8-22ced0717fb2 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:43.589556 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c1603-aa1f-45c3-adb8-22ced0717fb2 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:43.590037 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c1603-aa1f-45c3-adb8-22ced0717fb2 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:43.592029 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 74/141] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:23:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:43.598614 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f057fb-664f-4194-b750-90c3b1616e7c None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:43.599269 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f057fb-664f-4194-b750-90c3b1616e7c None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:43.599790 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f057fb-664f-4194-b750-90c3b1616e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:43.623859 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-60f057fb-664f-4194-b750-90c3b1616e7c None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:43.625497 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-60f057fb-664f-4194-b750-90c3b1616e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:43.677480 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 68/142] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 18 16:23:43 2022] POST /identity/v3/users => generated 283 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:23:43.741182 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 75/143] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:43.784483 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76332040-1505-4ead-8848-bd09c01b437c None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:43.849411 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd0de04-706c-4ddb-8abc-9e370277c2c2 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:43.850308 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd0de04-706c-4ddb-8abc-9e370277c2c2 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:43.850603 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd0de04-706c-4ddb-8abc-9e370277c2c2 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:43.852836 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 76/144] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:23:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:43.859361 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74f9ed-7d7f-4fc1-9957-19230a694801 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:43.859693 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74f9ed-7d7f-4fc1-9957-19230a694801 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:43.859973 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74f9ed-7d7f-4fc1-9957-19230a694801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:43.879554 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-bc74f9ed-7d7f-4fc1-9957-19230a694801 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:43.881011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-bc74f9ed-7d7f-4fc1-9957-19230a694801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:43.982482 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76332040-1505-4ead-8848-bd09c01b437c None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:44.034413 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 77/145] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:44.040794 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:44.054798 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1deb70c-9346-4672-8897-b3ff73671aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:44.069315 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1deb70c-9346-4672-8897-b3ff73671aa6 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:44.069315 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1deb70c-9346-4672-8897-b3ff73671aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:44.069315 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1deb70c-9346-4672-8897-b3ff73671aa6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:44.070457 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1deb70c-9346-4672-8897-b3ff73671aa6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:44.078817 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-c1deb70c-9346-4672-8897-b3ff73671aa6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:23:44.078817 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:44.078817 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:44.078817 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:44.078817 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:44.078817 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:44.078817 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:44.078817 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:44.078817 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:44.078817 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:44.078817 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:44.078817 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:44.078817 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:44.078817 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:44.078817 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:44.078817 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:44.079755 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:44.079755 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:44.079755 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:44.079755 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:44.079755 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:44.079755 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:44.079755 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:44.079755 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:44.079755 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:44.079755 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:44.079755 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:44.079755 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:44.079755 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:44.079755 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:44.079755 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:44.079755 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:44.080527 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:44.080527 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:44.080527 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:44.080527 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:44.080527 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:44.080527 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:44.080527 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:44.080527 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:44.080527 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:44.080527 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:44.080527 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:23:44.080527 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:23:44.081109 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 69/146] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 18 16:23:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:23:44.087400 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1deb70c-9346-4672-8897-b3ff73671aa6 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:44.088625 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc74f9ed-7d7f-4fc1-9957-19230a694801 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:44.092300 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:44.279575 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc74f9ed-7d7f-4fc1-9957-19230a694801 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:44.296338 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1deb70c-9346-4672-8897-b3ff73671aa6 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:44.340935 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670d1382-6b0e-4628-b475-68c913cb4adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:44.341760 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670d1382-6b0e-4628-b475-68c913cb4adf None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:44.342050 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670d1382-6b0e-4628-b475-68c913cb4adf None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:44.342324 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670d1382-6b0e-4628-b475-68c913cb4adf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:44.357319 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670d1382-6b0e-4628-b475-68c913cb4adf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:44.365256 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-670d1382-6b0e-4628-b475-68c913cb4adf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:23:44.365256 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:44.365256 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:44.365256 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:44.365256 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:44.365256 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:44.365256 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:44.365256 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:44.365256 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:44.365256 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:44.365256 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:44.365256 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:44.365256 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:44.365256 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:44.365256 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:44.365256 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:44.366130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:44.366130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:44.366130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:44.366130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:44.366130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:44.366130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:44.366130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:44.366130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:44.366130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:44.366130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:44.366130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:44.366130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:44.366130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:44.366130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:44.366130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:44.366130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:44.366899 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:44.366899 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:44.366899 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:44.366899 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:44.366899 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:44.366899 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:44.366899 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:44.366899 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:44.366899 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:44.366899 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:44.366899 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:23:44.366899 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:23:44.367473 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 78/147] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:23:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:23:44.373901 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670d1382-6b0e-4628-b475-68c913cb4adf None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:44.382557 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bbe712-1dc5-4785-a222-d85900e8e6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:44.383248 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bbe712-1dc5-4785-a222-d85900e8e6aa None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:44.384148 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bbe712-1dc5-4785-a222-d85900e8e6aa None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:44.384599 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bbe712-1dc5-4785-a222-d85900e8e6aa None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:44.385528 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bbe712-1dc5-4785-a222-d85900e8e6aa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:44.385835 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bbe712-1dc5-4785-a222-d85900e8e6aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:44.387237 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bbe712-1dc5-4785-a222-d85900e8e6aa None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:44.401987 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 70/148] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:23:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:23:44.409422 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bbe712-1dc5-4785-a222-d85900e8e6aa None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:44.461335 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:44.506956 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:44.529463 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670d1382-6b0e-4628-b475-68c913cb4adf None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:44.573575 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bbe712-1dc5-4785-a222-d85900e8e6aa None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:44.613201 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b95dbb-c724-47a1-8800-14bf807400b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:44.613891 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b95dbb-c724-47a1-8800-14bf807400b3 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:44.614294 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b95dbb-c724-47a1-8800-14bf807400b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:44.614574 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b95dbb-c724-47a1-8800-14bf807400b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:44.615335 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b95dbb-c724-47a1-8800-14bf807400b3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:44.615686 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b95dbb-c724-47a1-8800-14bf807400b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:44.617336 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b95dbb-c724-47a1-8800-14bf807400b3 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:44.617443 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344acb64-baa0-45c8-b199-86f97dc316bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:44.624714 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344acb64-baa0-45c8-b199-86f97dc316bc None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:44.624714 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344acb64-baa0-45c8-b199-86f97dc316bc None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:44.624714 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344acb64-baa0-45c8-b199-86f97dc316bc None admin] PATH_INFO: `/v3/users/e18df28d16ba462694258c7ad963170a` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:44.624714 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344acb64-baa0-45c8-b199-86f97dc316bc None admin] RBAC: Authorizing `identity:get_user(user_id=e18df28d16ba462694258c7ad963170a)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:44.624714 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344acb64-baa0-45c8-b199-86f97dc316bc None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:44.631356 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 71/149] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:44 2022] GET /identity/v3/users/e18df28d16ba462694258c7ad963170a => generated 247 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:44.632477 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344acb64-baa0-45c8-b199-86f97dc316bc None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:44.637630 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 79/150] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 18 16:23:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:44.648587 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b95dbb-c724-47a1-8800-14bf807400b3 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:44.805091 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344acb64-baa0-45c8-b199-86f97dc316bc None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:44.844139 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:44.870025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b95dbb-c724-47a1-8800-14bf807400b3 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:44.887807 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d2636b-4e95-4b53-8e78-b8a3459889dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:44.888845 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2636b-4e95-4b53-8e78-b8a3459889dc None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:44.889355 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2636b-4e95-4b53-8e78-b8a3459889dc None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:44.889832 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2636b-4e95-4b53-8e78-b8a3459889dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:44.924839 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:44.945135 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d8a6f7-1d65-4236-bfcb-2b360dcf57e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:44.947960 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d8a6f7-1d65-4236-bfcb-2b360dcf57e4 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:44.949345 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d8a6f7-1d65-4236-bfcb-2b360dcf57e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:44.951149 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d8a6f7-1d65-4236-bfcb-2b360dcf57e4 None admin] PATH_INFO: `/v3/users/cda57e656e834b9792465e79b138c9c4` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:44.952931 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d8a6f7-1d65-4236-bfcb-2b360dcf57e4 None admin] RBAC: Authorizing `identity:get_user(user_id=cda57e656e834b9792465e79b138c9c4)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:44.955410 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d8a6f7-1d65-4236-bfcb-2b360dcf57e4 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:44.960809 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 80/151] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:44 2022] GET /identity/v3/users/cda57e656e834b9792465e79b138c9c4 => generated 275 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:44.964104 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d8a6f7-1d65-4236-bfcb-2b360dcf57e4 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:45.020493 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d2636b-4e95-4b53-8e78-b8a3459889dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:45.021979 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d2636b-4e95-4b53-8e78-b8a3459889dc None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:45.140100 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 72/152] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:23:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:23:45.148360 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d2636b-4e95-4b53-8e78-b8a3459889dc None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:45.183879 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:45.255936 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d8a6f7-1d65-4236-bfcb-2b360dcf57e4 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:45.285535 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d2636b-4e95-4b53-8e78-b8a3459889dc None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:45.320244 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d361b78d-0156-4328-9d60-25f023473c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:45.320938 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361b78d-0156-4328-9d60-25f023473c83 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:45.321292 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361b78d-0156-4328-9d60-25f023473c83 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:45.321577 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361b78d-0156-4328-9d60-25f023473c83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:45.333398 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4284bbec-9e74-4638-9d75-af73d8869efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:45.334381 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4284bbec-9e74-4638-9d75-af73d8869efb None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:45.334669 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4284bbec-9e74-4638-9d75-af73d8869efb None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:45.334945 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4284bbec-9e74-4638-9d75-af73d8869efb None admin] PATH_INFO: `/v3/projects/04afdb2deaea49078a14a588c9d6de7e` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:45.336186 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4284bbec-9e74-4638-9d75-af73d8869efb None admin] RBAC: Authorizing `identity:get_project(project_id=04afdb2deaea49078a14a588c9d6de7e)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:45.338281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4284bbec-9e74-4638-9d75-af73d8869efb None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:45.340109 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 73/153] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:23:45 2022] GET /identity/v3/projects/04afdb2deaea49078a14a588c9d6de7e => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:45.358459 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06be253-4f05-4a55-b4e7-4c0ce8a2065d None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:45.358751 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06be253-4f05-4a55-b4e7-4c0ce8a2065d None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:45.359025 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06be253-4f05-4a55-b4e7-4c0ce8a2065d None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:45.360713 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 74/154] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:23:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:45.366141 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e2fcce-48e8-4042-b3c5-283929a8e5fa None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:45.366433 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e2fcce-48e8-4042-b3c5-283929a8e5fa None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:45.366713 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e2fcce-48e8-4042-b3c5-283929a8e5fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:45.388678 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-b1e2fcce-48e8-4042-b3c5-283929a8e5fa None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:45.389887 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-b1e2fcce-48e8-4042-b3c5-283929a8e5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:45.408258 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:45.418434 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d361b78d-0156-4328-9d60-25f023473c83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:45.419975 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d361b78d-0156-4328-9d60-25f023473c83 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:45.513881 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 75/155] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:23:45.514755 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e2fcce-48e8-4042-b3c5-283929a8e5fa None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:45.547162 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:45.612042 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 81/156] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:23:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 648 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:23:45.612959 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d361b78d-0156-4328-9d60-25f023473c83 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:45.685678 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e2fcce-48e8-4042-b3c5-283929a8e5fa None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:45.759016 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:45.759201 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7eef1d-7ca8-4d23-b7ed-cb69b403de51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:45.760374 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7eef1d-7ca8-4d23-b7ed-cb69b403de51 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:45.760660 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7eef1d-7ca8-4d23-b7ed-cb69b403de51 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:45.760944 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7eef1d-7ca8-4d23-b7ed-cb69b403de51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:45.761783 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7eef1d-7ca8-4d23-b7ed-cb69b403de51 None admin] RBAC: Adding query filter params (role.id=98eaadb3669f4b608d572c001742f559, scope.project.id=04afdb2deaea49078a14a588c9d6de7e, user.id=e18df28d16ba462694258c7ad963170a) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:45.762091 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7eef1d-7ca8-4d23-b7ed-cb69b403de51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:45.767615 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7eef1d-7ca8-4d23-b7ed-cb69b403de51 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:45.767832 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d361b78d-0156-4328-9d60-25f023473c83 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:45.785778 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 76/157] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 18 16:23:45 2022] GET /identity/v3/role_assignments?user.id=e18df28d16ba462694258c7ad963170a&scope.project.id=04afdb2deaea49078a14a588c9d6de7e&role.id=98eaadb3669f4b608d572c001742f559 => generated 608 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:45.786608 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7eef1d-7ca8-4d23-b7ed-cb69b403de51 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:45.830182 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3e055c-799d-403a-bf22-5c5451f1c1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:45.830594 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e055c-799d-403a-bf22-5c5451f1c1ef None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:45.830886 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e055c-799d-403a-bf22-5c5451f1c1ef None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:45.831169 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e055c-799d-403a-bf22-5c5451f1c1ef None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:45.858237 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3e055c-799d-403a-bf22-5c5451f1c1ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:45.874882 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-1d3e055c-799d-403a-bf22-5c5451f1c1ef None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:23:45.874882 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:45.874882 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:45.874882 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:45.874882 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:45.874882 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:45.874882 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:45.874882 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:45.874882 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:45.874882 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:45.874882 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:45.874882 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:45.874882 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:45.874882 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:45.874882 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:45.874882 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:45.875855 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:45.875855 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:45.875855 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:45.875855 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:45.875855 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:45.875855 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:45.875855 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:45.875855 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:45.875855 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:45.875855 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:45.875855 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:45.875855 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:45.875855 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:45.875855 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:45.875855 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:45.875855 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:45.876633 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:45.876633 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:45.876633 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:45.876633 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:45.876633 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:45.876633 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:45.876633 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:45.876633 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:45.876633 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:45.876633 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:45.876633 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:23:45.876633 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:23:45.876633 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 82/158] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 18 16:23:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:23:45.883758 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3e055c-799d-403a-bf22-5c5451f1c1ef None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:45.957305 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:45.984636 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7eef1d-7ca8-4d23-b7ed-cb69b403de51 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:46.042401 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3e055c-799d-403a-bf22-5c5451f1c1ef None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:46.051012 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d259167-705c-415a-aa8c-ed823f1a4290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:46.052002 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d259167-705c-415a-aa8c-ed823f1a4290 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:46.052281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d259167-705c-415a-aa8c-ed823f1a4290 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:46.052549 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d259167-705c-415a-aa8c-ed823f1a4290 None admin] PATH_INFO: `/v3/projects/04afdb2deaea49078a14a588c9d6de7e` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:46.053844 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d259167-705c-415a-aa8c-ed823f1a4290 None admin] RBAC: Authorizing `identity:get_project(project_id=04afdb2deaea49078a14a588c9d6de7e)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:46.055337 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d259167-705c-415a-aa8c-ed823f1a4290 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:46.058150 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 77/159] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:23:45 2022] GET /identity/v3/projects/04afdb2deaea49078a14a588c9d6de7e => generated 298 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:46.064447 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d259167-705c-415a-aa8c-ed823f1a4290 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:46.124257 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0d0d0e-725f-4264-9d4e-6f541fadfc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:46.124964 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d0d0e-725f-4264-9d4e-6f541fadfc8a None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:46.125255 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d0d0e-725f-4264-9d4e-6f541fadfc8a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:46.125542 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d0d0e-725f-4264-9d4e-6f541fadfc8a None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:46.126309 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0d0d0e-725f-4264-9d4e-6f541fadfc8a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:46.126625 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0d0d0e-725f-4264-9d4e-6f541fadfc8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:46.128088 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0d0d0e-725f-4264-9d4e-6f541fadfc8a None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:46.143811 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 83/160] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:23:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:46.152521 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0d0d0e-725f-4264-9d4e-6f541fadfc8a None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:46.180070 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:46.226017 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d259167-705c-415a-aa8c-ed823f1a4290 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:46.288239 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec15b66-b8a6-47a7-bd51-06838724c3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:46.288923 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec15b66-b8a6-47a7-bd51-06838724c3d0 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:46.289227 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec15b66-b8a6-47a7-bd51-06838724c3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:46.289480 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec15b66-b8a6-47a7-bd51-06838724c3d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:46.295438 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec15b66-b8a6-47a7-bd51-06838724c3d0 None admin] RBAC: Adding query filter params (role.id=aa0552e8ba924f5d9cb1740da7d4a4dc, scope.project.id=04afdb2deaea49078a14a588c9d6de7e, user.id=cda57e656e834b9792465e79b138c9c4) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:46.295438 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec15b66-b8a6-47a7-bd51-06838724c3d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:46.295438 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec15b66-b8a6-47a7-bd51-06838724c3d0 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:46.300327 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 78/161] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 18 16:23:46 2022] GET /identity/v3/role_assignments?user.id=cda57e656e834b9792465e79b138c9c4&scope.project.id=04afdb2deaea49078a14a588c9d6de7e&role.id=aa0552e8ba924f5d9cb1740da7d4a4dc => generated 608 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:46.313816 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0d0d0e-725f-4264-9d4e-6f541fadfc8a None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:46.380328 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d833df-211b-4573-a438-aa2f64a829bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:46.381636 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d833df-211b-4573-a438-aa2f64a829bd None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:46.382141 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d833df-211b-4573-a438-aa2f64a829bd None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:46.382654 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d833df-211b-4573-a438-aa2f64a829bd None admin] PATH_INFO: `/v3/users/cda57e656e834b9792465e79b138c9c4` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:46.384521 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d833df-211b-4573-a438-aa2f64a829bd None admin] RBAC: Authorizing `identity:get_user(user_id=cda57e656e834b9792465e79b138c9c4)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:46.386617 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d833df-211b-4573-a438-aa2f64a829bd None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:46.387233 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d530ad20-404e-425c-b829-c3900f88fdb9 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:46.387553 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d530ad20-404e-425c-b829-c3900f88fdb9 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:46.387842 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d530ad20-404e-425c-b829-c3900f88fdb9 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:46.389315 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 79/162] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:23:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:23:46.389723 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 84/163] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:46 2022] GET /identity/v3/users/cda57e656e834b9792465e79b138c9c4 => generated 275 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:46.394796 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d530ad20-404e-425c-b829-c3900f88fdb9 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:46.400405 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34678a8-3a1b-43de-bcc2-444c7f3e6d6e None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:46.400939 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34678a8-3a1b-43de-bcc2-444c7f3e6d6e None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:46.401477 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34678a8-3a1b-43de-bcc2-444c7f3e6d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:46.411521 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:46.432955 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-d34678a8-3a1b-43de-bcc2-444c7f3e6d6e None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:46.434518 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d34678a8-3a1b-43de-bcc2-444c7f3e6d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:46.563084 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 85/164] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:23:46.600848 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d530ad20-404e-425c-b829-c3900f88fdb9 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:46.620415 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34678a8-3a1b-43de-bcc2-444c7f3e6d6e None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:46.652873 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:46.672330 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e2d730-80e7-43e8-9f3e-309027f3fc6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:46.673014 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2d730-80e7-43e8-9f3e-309027f3fc6d None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:46.673298 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2d730-80e7-43e8-9f3e-309027f3fc6d None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:46.673575 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2d730-80e7-43e8-9f3e-309027f3fc6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:46.776800 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34678a8-3a1b-43de-bcc2-444c7f3e6d6e None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:46.819100 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811a2790-467c-4f2f-a887-6c338c70cbbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:46.820359 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811a2790-467c-4f2f-a887-6c338c70cbbc None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:46.820874 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811a2790-467c-4f2f-a887-6c338c70cbbc None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:46.821386 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811a2790-467c-4f2f-a887-6c338c70cbbc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:46.846439 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811a2790-467c-4f2f-a887-6c338c70cbbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:46.857605 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-811a2790-467c-4f2f-a887-6c338c70cbbc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:23:46.857605 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:46.857605 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:46.857605 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:46.857605 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:46.857605 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:46.857605 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:46.857605 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:46.857605 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:46.857605 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:46.857605 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:46.857605 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:46.857605 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:46.857605 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:46.857605 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:46.857605 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:46.858503 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:46.858503 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:46.858503 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:46.858503 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:46.858503 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:46.858503 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:46.858503 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:46.858503 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:46.858503 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:46.858503 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:46.858503 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:46.858503 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:46.858503 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:46.858503 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:46.858503 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:46.858503 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:46.859505 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:46.859505 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:46.859505 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:46.859505 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:46.859505 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:46.859505 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:46.859505 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:46.859505 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:46.859505 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:46.859505 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:46.859505 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:23:46.859505 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:23:46.861496 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 86/165] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:23:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:23:46.872471 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811a2790-467c-4f2f-a887-6c338c70cbbc None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:46.879228 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e2d730-80e7-43e8-9f3e-309027f3fc6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:46.880809 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e2d730-80e7-43e8-9f3e-309027f3fc6d None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:46.906072 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:46.998628 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:47.038224 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811a2790-467c-4f2f-a887-6c338c70cbbc None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:47.086421 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada69c5a-c3de-48f0-98a2-4f612291247b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:47.088408 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada69c5a-c3de-48f0-98a2-4f612291247b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:47.088922 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada69c5a-c3de-48f0-98a2-4f612291247b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:47.089405 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada69c5a-c3de-48f0-98a2-4f612291247b None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:47.090368 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada69c5a-c3de-48f0-98a2-4f612291247b None admin] RBAC: Adding query filter params (name=member) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:47.090883 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada69c5a-c3de-48f0-98a2-4f612291247b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:47.093125 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada69c5a-c3de-48f0-98a2-4f612291247b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:47.105398 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 87/166] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 18 16:23:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:23:47.107041 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd20b9a-a1d8-410d-a40c-e30637900517 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:47.108146 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd20b9a-a1d8-410d-a40c-e30637900517 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:47.108647 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd20b9a-a1d8-410d-a40c-e30637900517 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:47.110165 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 88/167] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:23:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:47.111190 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd20b9a-a1d8-410d-a40c-e30637900517 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:47.151889 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 80/168] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:23:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 757 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:23:47.154644 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ebc02-049e-4c4c-b17c-2adf064ba75b None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:47.154644 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ebc02-049e-4c4c-b17c-2adf064ba75b None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:47.154644 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ebc02-049e-4c4c-b17c-2adf064ba75b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:47.183367 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-6c1ebc02-049e-4c4c-b17c-2adf064ba75b None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:47.184603 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-6c1ebc02-049e-4c4c-b17c-2adf064ba75b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:47.276949 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd20b9a-a1d8-410d-a40c-e30637900517 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:47.318795 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b9cf9-6a00-4975-b7ed-1879990a8371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:47.320851 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b9cf9-6a00-4975-b7ed-1879990a8371 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:47.321948 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b9cf9-6a00-4975-b7ed-1879990a8371 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:47.322792 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b9cf9-6a00-4975-b7ed-1879990a8371 None admin] PATH_INFO: `/v3/users/cda57e656e834b9792465e79b138c9c4` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:47.324335 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063b9cf9-6a00-4975-b7ed-1879990a8371 None admin] RBAC: Authorizing `identity:get_user(user_id=cda57e656e834b9792465e79b138c9c4)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:47.326135 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063b9cf9-6a00-4975-b7ed-1879990a8371 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:47.328296 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 89/169] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:47 2022] GET /identity/v3/users/cda57e656e834b9792465e79b138c9c4 => generated 275 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:47.330306 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b9cf9-6a00-4975-b7ed-1879990a8371 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:47.334963 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:47.342544 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 81/170] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:47.343404 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ebc02-049e-4c4c-b17c-2adf064ba75b None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:47.397471 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:47.465307 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ebc02-049e-4c4c-b17c-2adf064ba75b None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:47.502965 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dae824a-ad33-4cb1-8f36-dcd363f9add3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:47.504034 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae824a-ad33-4cb1-8f36-dcd363f9add3 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:47.504596 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae824a-ad33-4cb1-8f36-dcd363f9add3 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:47.505080 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae824a-ad33-4cb1-8f36-dcd363f9add3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:47.582137 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dae824a-ad33-4cb1-8f36-dcd363f9add3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:47.584048 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dae824a-ad33-4cb1-8f36-dcd363f9add3 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:47.586256 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b9cf9-6a00-4975-b7ed-1879990a8371 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:47.630019 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:47.676831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7861aa6b-117e-43db-9211-1d3e741cd54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:47.677670 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7861aa6b-117e-43db-9211-1d3e741cd54f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:47.677956 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7861aa6b-117e-43db-9211-1d3e741cd54f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:47.678234 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7861aa6b-117e-43db-9211-1d3e741cd54f None admin] PATH_INFO: `/v3/projects/04afdb2deaea49078a14a588c9d6de7e` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:47.684368 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7861aa6b-117e-43db-9211-1d3e741cd54f None admin] RBAC: Authorizing `identity:get_project(project_id=04afdb2deaea49078a14a588c9d6de7e)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:47.686752 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7861aa6b-117e-43db-9211-1d3e741cd54f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:47.688554 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 90/171] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:23:47 2022] GET /identity/v3/projects/04afdb2deaea49078a14a588c9d6de7e => generated 298 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:47.689330 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7861aa6b-117e-43db-9211-1d3e741cd54f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:47.702505 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 82/172] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:23:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:23:47.703684 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dae824a-ad33-4cb1-8f36-dcd363f9add3 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:47.813343 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dae824a-ad33-4cb1-8f36-dcd363f9add3 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:47.827970 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:47.839676 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7861aa6b-117e-43db-9211-1d3e741cd54f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:47.863055 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6730028b-010a-4311-ab22-a5caf11a1201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:47.863833 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6730028b-010a-4311-ab22-a5caf11a1201 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:47.864122 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6730028b-010a-4311-ab22-a5caf11a1201 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:47.864406 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6730028b-010a-4311-ab22-a5caf11a1201 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:47.865464 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6730028b-010a-4311-ab22-a5caf11a1201 None admin] RBAC: Adding query filter params (role.id=729cd9c0f42d47b6a8eee01e5f5a2d28, scope.project.id=04afdb2deaea49078a14a588c9d6de7e, user.id=cda57e656e834b9792465e79b138c9c4) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:47.865781 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6730028b-010a-4311-ab22-a5caf11a1201 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:47.867420 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6730028b-010a-4311-ab22-a5caf11a1201 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:47.874757 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 83/173] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 18 16:23:47 2022] GET /identity/v3/role_assignments?user.id=cda57e656e834b9792465e79b138c9c4&scope.project.id=04afdb2deaea49078a14a588c9d6de7e&role.id=729cd9c0f42d47b6a8eee01e5f5a2d28 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:47.875662 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6730028b-010a-4311-ab22-a5caf11a1201 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:47.878234 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:47.929512 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d340c37-a5b7-4704-ae6d-d4cf7f7f32fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:47.930388 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d340c37-a5b7-4704-ae6d-d4cf7f7f32fc None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:47.931247 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d340c37-a5b7-4704-ae6d-d4cf7f7f32fc None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:47.932332 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d340c37-a5b7-4704-ae6d-d4cf7f7f32fc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:47.960491 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d340c37-a5b7-4704-ae6d-d4cf7f7f32fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:47.975527 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-0d340c37-a5b7-4704-ae6d-d4cf7f7f32fc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:23:47.975527 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:47.975527 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:47.975527 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:47.975527 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:47.975527 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:47.975527 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:47.975527 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:47.975527 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:47.975527 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:47.975527 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:47.975527 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:47.975527 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:47.975527 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:47.975527 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:47.975527 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:47.976505 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:47.976505 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:47.976505 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:47.976505 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:47.976505 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:47.976505 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:47.976505 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:47.976505 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:47.976505 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:47.976505 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:47.976505 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:47.976505 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:47.976505 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:47.976505 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:47.976505 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:47.976505 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:47.977358 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:47.977358 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:47.977358 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:47.977358 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:47.977358 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:47.977358 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:47.977358 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:47.977358 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:47.977358 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:47.977358 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:47.977358 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:23:47.977358 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:23:47.981146 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 91/174] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:23:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:23:47.987745 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d340c37-a5b7-4704-ae6d-d4cf7f7f32fc None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:47.997320 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6730028b-010a-4311-ab22-a5caf11a1201 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:48.035928 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa554dbf-1852-42be-9f07-89f7ecfec024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:48.036879 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa554dbf-1852-42be-9f07-89f7ecfec024 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:48.037165 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa554dbf-1852-42be-9f07-89f7ecfec024 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:48.037444 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa554dbf-1852-42be-9f07-89f7ecfec024 None admin] PATH_INFO: `/v3/projects/277b079204de49d98ea603246c8a67c6` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:48.038783 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa554dbf-1852-42be-9f07-89f7ecfec024 None admin] RBAC: Authorizing `identity:get_project(project_id=277b079204de49d98ea603246c8a67c6)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:48.040904 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa554dbf-1852-42be-9f07-89f7ecfec024 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:48.042900 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 84/175] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:23:47 2022] GET /identity/v3/projects/277b079204de49d98ea603246c8a67c6 => generated 312 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:48.044341 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692edd1-02e4-4430-8eb4-5fa10736ff93 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:48.044629 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692edd1-02e4-4430-8eb4-5fa10736ff93 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:48.044987 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692edd1-02e4-4430-8eb4-5fa10736ff93 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:48.046353 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 85/176] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:23:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:48.049118 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6692edd1-02e4-4430-8eb4-5fa10736ff93 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:48.144860 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:48.191101 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6692edd1-02e4-4430-8eb4-5fa10736ff93 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:48.195114 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d340c37-a5b7-4704-ae6d-d4cf7f7f32fc None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:48.230160 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:48.235683 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f7a5d5-b742-49d0-9907-9cb6bfba194f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:48.235849 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca53404-a415-4a43-b765-7492bf895e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:48.236361 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7a5d5-b742-49d0-9907-9cb6bfba194f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:48.236462 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca53404-a415-4a43-b765-7492bf895e92 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:48.236645 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7a5d5-b742-49d0-9907-9cb6bfba194f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:48.236737 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca53404-a415-4a43-b765-7492bf895e92 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:48.236910 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7a5d5-b742-49d0-9907-9cb6bfba194f None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:48.237008 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca53404-a415-4a43-b765-7492bf895e92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:48.237650 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f7a5d5-b742-49d0-9907-9cb6bfba194f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:48.237810 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca53404-a415-4a43-b765-7492bf895e92 None admin] RBAC: Adding query filter params (role.id=aa0552e8ba924f5d9cb1740da7d4a4dc, scope.project.id=277b079204de49d98ea603246c8a67c6, user.id=cda57e656e834b9792465e79b138c9c4) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:48.237963 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f7a5d5-b742-49d0-9907-9cb6bfba194f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:48.238111 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca53404-a415-4a43-b765-7492bf895e92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:48.239335 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f7a5d5-b742-49d0-9907-9cb6bfba194f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:48.241857 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca53404-a415-4a43-b765-7492bf895e92 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:48.250864 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 92/177] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 18 16:23:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:48.255392 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebaa6cf-00df-4242-8928-d94246290e85 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:48.255392 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebaa6cf-00df-4242-8928-d94246290e85 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:48.255392 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebaa6cf-00df-4242-8928-d94246290e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:48.256048 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 86/178] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 18 16:23:48 2022] GET /identity/v3/role_assignments?user.id=cda57e656e834b9792465e79b138c9c4&scope.project.id=277b079204de49d98ea603246c8a67c6&role.id=aa0552e8ba924f5d9cb1740da7d4a4dc => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:48.257761 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbda0ba7-dac9-498c-96bc-d24cac1c71e8 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:48.258059 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbda0ba7-dac9-498c-96bc-d24cac1c71e8 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:48.258343 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbda0ba7-dac9-498c-96bc-d24cac1c71e8 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:48.259787 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 87/179] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:23:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:48.260490 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbda0ba7-dac9-498c-96bc-d24cac1c71e8 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:48.284269 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-9ebaa6cf-00df-4242-8928-d94246290e85 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:48.285785 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-9ebaa6cf-00df-4242-8928-d94246290e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:48.385902 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 93/180] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:48.388092 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b1cea0-4f16-4640-96a6-9a4917398822 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:48.388586 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b1cea0-4f16-4640-96a6-9a4917398822 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:48.389031 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b1cea0-4f16-4640-96a6-9a4917398822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:48.424743 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-b8b1cea0-4f16-4640-96a6-9a4917398822 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:48.426340 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-b8b1cea0-4f16-4640-96a6-9a4917398822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:48.454130 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbda0ba7-dac9-498c-96bc-d24cac1c71e8 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:48.501608 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcc27f5-0ecb-4355-9828-7a309d81501f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:48.502604 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc27f5-0ecb-4355-9828-7a309d81501f None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:48.502972 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc27f5-0ecb-4355-9828-7a309d81501f None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:48.503177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc27f5-0ecb-4355-9828-7a309d81501f None admin] PATH_INFO: `/v3/users/77e423c38dcf4d16a61998f6eb9e4ca3` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:48.510139 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcc27f5-0ecb-4355-9828-7a309d81501f None admin] RBAC: Authorizing `identity:get_user(user_id=77e423c38dcf4d16a61998f6eb9e4ca3)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:48.512107 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcc27f5-0ecb-4355-9828-7a309d81501f None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:48.514059 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 88/181] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:48 2022] GET /identity/v3/users/77e423c38dcf4d16a61998f6eb9e4ca3 => generated 289 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:48.515078 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcc27f5-0ecb-4355-9828-7a309d81501f None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:48.518531 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:48.524720 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 94/182] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:48.527581 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e9f9be-704e-473b-b430-04698f2a87cb None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:48.527917 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e9f9be-704e-473b-b430-04698f2a87cb None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:48.528196 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e9f9be-704e-473b-b430-04698f2a87cb None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:48.529676 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 95/183] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:23:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:48.530285 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e9f9be-704e-473b-b430-04698f2a87cb None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:48.599422 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:48.643199 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcc27f5-0ecb-4355-9828-7a309d81501f None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:48.647896 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e9f9be-704e-473b-b430-04698f2a87cb None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:48.683166 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd256dd4-b073-4ddc-b849-f11879c0fe43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:48.684365 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd256dd4-b073-4ddc-b849-f11879c0fe43 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:48.684858 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd256dd4-b073-4ddc-b849-f11879c0fe43 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:48.685371 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd256dd4-b073-4ddc-b849-f11879c0fe43 None admin] PATH_INFO: `/v3/projects/04afdb2deaea49078a14a588c9d6de7e` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:48.686952 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd256dd4-b073-4ddc-b849-f11879c0fe43 None admin] RBAC: Authorizing `identity:get_project(project_id=04afdb2deaea49078a14a588c9d6de7e)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:48.688479 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd256dd4-b073-4ddc-b849-f11879c0fe43 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:48.690234 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 96/184] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:23:48 2022] GET /identity/v3/projects/04afdb2deaea49078a14a588c9d6de7e => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:48.691427 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481bc58-17a9-4dd1-9cc5-8411f701af2a None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:48.691741 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481bc58-17a9-4dd1-9cc5-8411f701af2a None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:48.692021 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481bc58-17a9-4dd1-9cc5-8411f701af2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:48.698652 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10ff45a-0d97-443f-97d1-c9f2f21525c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:48.699474 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ff45a-0d97-443f-97d1-c9f2f21525c2 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:48.699798 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ff45a-0d97-443f-97d1-c9f2f21525c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:48.700075 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ff45a-0d97-443f-97d1-c9f2f21525c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:48.712462 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10ff45a-0d97-443f-97d1-c9f2f21525c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:48.713207 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-c481bc58-17a9-4dd1-9cc5-8411f701af2a None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:48.714563 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-c481bc58-17a9-4dd1-9cc5-8411f701af2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:48.720533 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-f10ff45a-0d97-443f-97d1-c9f2f21525c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:23:48.720533 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:48.720533 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:48.720533 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:48.720533 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:48.720533 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:48.720533 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:48.720533 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:48.720533 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:48.720533 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:48.720533 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:48.720533 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:48.720533 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:48.720533 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:48.720533 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:48.720533 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:48.721441 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:48.721441 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:48.721441 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:48.721441 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:48.721441 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:48.721441 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:48.721441 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:48.721441 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:48.721441 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:48.721441 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:48.721441 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:48.721441 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:48.721441 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:48.721441 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:48.721441 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:48.721441 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:48.722235 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:48.722235 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:48.722235 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:48.722235 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:48.722235 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:48.722235 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:48.722235 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:48.722235 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:48.722235 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:48.722235 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:48.722235 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:23:48.722235 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:23:48.722235 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 89/185] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 18 16:23:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:23:48.722875 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10ff45a-0d97-443f-97d1-c9f2f21525c2 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:48.792744 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:48.795147 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 97/186] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:48.819626 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c481bc58-17a9-4dd1-9cc5-8411f701af2a None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:48.839585 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10ff45a-0d97-443f-97d1-c9f2f21525c2 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:48.857131 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:48.877924 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e663b9-f37f-4e2c-8163-8c3e5e8378e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:48.878969 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e663b9-f37f-4e2c-8163-8c3e5e8378e7 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:48.879258 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e663b9-f37f-4e2c-8163-8c3e5e8378e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:48.879569 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e663b9-f37f-4e2c-8163-8c3e5e8378e7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:48.891229 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e663b9-f37f-4e2c-8163-8c3e5e8378e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:48.900334 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-95e663b9-f37f-4e2c-8163-8c3e5e8378e7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:23:48.900334 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:48.900334 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:48.900334 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:48.900334 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:48.900334 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:48.900334 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:48.900334 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:48.900334 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:48.900334 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:48.900334 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:48.900334 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:48.900334 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:48.900334 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:48.900334 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:48.900334 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:48.901281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:48.901281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:48.901281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:48.901281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:48.901281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:48.901281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:48.901281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:48.901281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:48.901281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:48.901281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:48.901281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:48.901281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:48.901281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:48.901281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:48.901281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:48.901281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:48.902308 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:48.902308 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:48.902308 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:48.902308 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:48.902308 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:48.902308 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:48.902308 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:48.902308 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:48.902308 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:48.902308 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:48.902308 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:23:48.902308 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:23:48.902308 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 90/187] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 18 16:23:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:23:48.902952 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e663b9-f37f-4e2c-8163-8c3e5e8378e7 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:48.911876 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c481bc58-17a9-4dd1-9cc5-8411f701af2a None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:48.959321 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133b5a81-08ca-4852-8c2d-e8396bfb31a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:48.960245 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b5a81-08ca-4852-8c2d-e8396bfb31a4 None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:48.960524 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b5a81-08ca-4852-8c2d-e8396bfb31a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:48.960798 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b5a81-08ca-4852-8c2d-e8396bfb31a4 None admin] PATH_INFO: `/v3/projects/04afdb2deaea49078a14a588c9d6de7e/users/77e423c38dcf4d16a61998f6eb9e4ca3/roles/2ba8d51abb6e4efa872c82ac8c9ba4a8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:48.963693 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133b5a81-08ca-4852-8c2d-e8396bfb31a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=04afdb2deaea49078a14a588c9d6de7e, user_id=77e423c38dcf4d16a61998f6eb9e4ca3, role_id=2ba8d51abb6e4efa872c82ac8c9ba4a8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:48.965076 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133b5a81-08ca-4852-8c2d-e8396bfb31a4 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:49.015577 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e663b9-f37f-4e2c-8163-8c3e5e8378e7 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:49.041137 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:49.043362 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 98/188] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 18 16:23:48 2022] PUT /identity/v3/projects/04afdb2deaea49078a14a588c9d6de7e/users/77e423c38dcf4d16a61998f6eb9e4ca3/roles/2ba8d51abb6e4efa872c82ac8c9ba4a8 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:23:49.053544 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133b5a81-08ca-4852-8c2d-e8396bfb31a4 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:49.065650 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd7cba-f81f-40b7-8496-3215050a4db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:49.066355 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd7cba-f81f-40b7-8496-3215050a4db1 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:49.066644 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd7cba-f81f-40b7-8496-3215050a4db1 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:49.066927 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd7cba-f81f-40b7-8496-3215050a4db1 None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:49.067951 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfd7cba-f81f-40b7-8496-3215050a4db1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:49.068265 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfd7cba-f81f-40b7-8496-3215050a4db1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:49.069730 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfd7cba-f81f-40b7-8496-3215050a4db1 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:49.082071 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 91/189] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:23:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:49.088286 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd7cba-f81f-40b7-8496-3215050a4db1 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:49.184052 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133b5a81-08ca-4852-8c2d-e8396bfb31a4 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:49.202835 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:49.228098 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd7cba-f81f-40b7-8496-3215050a4db1 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:49.237746 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684697d6-34f8-43d5-ac13-9a5008a10a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:49.238720 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684697d6-34f8-43d5-ac13-9a5008a10a64 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:49.239387 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684697d6-34f8-43d5-ac13-9a5008a10a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:49.240485 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684697d6-34f8-43d5-ac13-9a5008a10a64 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:49.242044 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684697d6-34f8-43d5-ac13-9a5008a10a64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:49.245328 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684697d6-34f8-43d5-ac13-9a5008a10a64 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:49.251764 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 99/190] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:23:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:49.251764 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684697d6-34f8-43d5-ac13-9a5008a10a64 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:49.284468 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deebb3b8-e6d3-45e9-abc6-0537c912398f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:49.285644 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deebb3b8-e6d3-45e9-abc6-0537c912398f None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:49.286130 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deebb3b8-e6d3-45e9-abc6-0537c912398f None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:49.286418 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deebb3b8-e6d3-45e9-abc6-0537c912398f None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:49.287286 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deebb3b8-e6d3-45e9-abc6-0537c912398f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:49.287567 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deebb3b8-e6d3-45e9-abc6-0537c912398f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:49.289061 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deebb3b8-e6d3-45e9-abc6-0537c912398f None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:49.302170 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 92/191] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:23:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:49.302820 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deebb3b8-e6d3-45e9-abc6-0537c912398f None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:49.368039 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:49.383081 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684697d6-34f8-43d5-ac13-9a5008a10a64 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:49.422430 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0815724-7e40-4e9a-adec-e8945b82978f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:49.423380 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0815724-7e40-4e9a-adec-e8945b82978f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:49.424729 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0815724-7e40-4e9a-adec-e8945b82978f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:49.424729 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0815724-7e40-4e9a-adec-e8945b82978f None admin] PATH_INFO: `/v3/users/993726e7f21a4c3eb6bb68bde8225406` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:49.432782 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deebb3b8-e6d3-45e9-abc6-0537c912398f None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:49.441568 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0815724-7e40-4e9a-adec-e8945b82978f None admin] RBAC: Authorizing `identity:get_user(user_id=993726e7f21a4c3eb6bb68bde8225406)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:49.443126 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0815724-7e40-4e9a-adec-e8945b82978f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:49.444933 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 100/192] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:49 2022] GET /identity/v3/users/993726e7f21a4c3eb6bb68bde8225406 => generated 283 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:49.450787 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0815724-7e40-4e9a-adec-e8945b82978f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:49.469782 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e7308c-83b0-4ceb-9b67-2205fcbc3f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:49.470465 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7308c-83b0-4ceb-9b67-2205fcbc3f38 None admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:49.470819 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7308c-83b0-4ceb-9b67-2205fcbc3f38 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:49.471021 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7308c-83b0-4ceb-9b67-2205fcbc3f38 None admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:49.471771 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e7308c-83b0-4ceb-9b67-2205fcbc3f38 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:49.473232 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e7308c-83b0-4ceb-9b67-2205fcbc3f38 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:49.491420 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:49.554558 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0815724-7e40-4e9a-adec-e8945b82978f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:49.571152 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 93/193] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 18 16:23:49 2022] POST /identity/v3/users => generated 297 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:23:49.573068 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e7308c-83b0-4ceb-9b67-2205fcbc3f38 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:49.606084 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709d836d-5da0-4792-99a6-de1c9f7c82b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:49.606998 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d836d-5da0-4792-99a6-de1c9f7c82b6 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:49.607341 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d836d-5da0-4792-99a6-de1c9f7c82b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:49.607658 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d836d-5da0-4792-99a6-de1c9f7c82b6 None admin] PATH_INFO: `/v3/users/993726e7f21a4c3eb6bb68bde8225406` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:49.609081 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709d836d-5da0-4792-99a6-de1c9f7c82b6 None admin] RBAC: Authorizing `identity:get_user(user_id=993726e7f21a4c3eb6bb68bde8225406)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:49.610722 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709d836d-5da0-4792-99a6-de1c9f7c82b6 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:49.612509 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 101/194] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:49 2022] GET /identity/v3/users/993726e7f21a4c3eb6bb68bde8225406 => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:49.619012 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709d836d-5da0-4792-99a6-de1c9f7c82b6 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:49.745293 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e7308c-83b0-4ceb-9b67-2205fcbc3f38 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:49.748794 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:49.753054 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709d836d-5da0-4792-99a6-de1c9f7c82b6 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:49.764453 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:49.792458 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5214ab7a-a194-4916-8f47-7b84826ec813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:49.793265 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5214ab7a-a194-4916-8f47-7b84826ec813 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:49.793654 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5214ab7a-a194-4916-8f47-7b84826ec813 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:49.794032 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5214ab7a-a194-4916-8f47-7b84826ec813 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:49.799288 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3261e787-198a-4613-84c7-e6014187e826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:49.800245 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3261e787-198a-4613-84c7-e6014187e826 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:49.800688 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3261e787-198a-4613-84c7-e6014187e826 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:49.800967 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3261e787-198a-4613-84c7-e6014187e826 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:49.903480 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3261e787-198a-4613-84c7-e6014187e826 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:49.904982 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3261e787-198a-4613-84c7-e6014187e826 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:49.909750 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5214ab7a-a194-4916-8f47-7b84826ec813 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:49.911438 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5214ab7a-a194-4916-8f47-7b84826ec813 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:50.041671 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 102/195] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:23:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:23:50.046989 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3261e787-198a-4613-84c7-e6014187e826 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:50.052594 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 94/196] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:23:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:23:50.056435 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:50.058869 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5214ab7a-a194-4916-8f47-7b84826ec813 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:50.062504 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:50.189985 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3261e787-198a-4613-84c7-e6014187e826 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:50.239936 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468663d0-90d9-4193-8666-e8c779d136fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:50.240804 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468663d0-90d9-4193-8666-e8c779d136fe None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:50.241900 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468663d0-90d9-4193-8666-e8c779d136fe None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:50.242842 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468663d0-90d9-4193-8666-e8c779d136fe None admin] PATH_INFO: `/v3/projects/cc6e57482c554178aa9b830aa0b83a96` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:50.244313 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468663d0-90d9-4193-8666-e8c779d136fe None admin] RBAC: Authorizing `identity:get_project(project_id=cc6e57482c554178aa9b830aa0b83a96)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:50.245815 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468663d0-90d9-4193-8666-e8c779d136fe None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:50.247597 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 103/197] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:23:50 2022] GET /identity/v3/projects/cc6e57482c554178aa9b830aa0b83a96 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:23:50.259296 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5214ab7a-a194-4916-8f47-7b84826ec813 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:50.262969 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468663d0-90d9-4193-8666-e8c779d136fe None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:50.337759 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2ee685-51d2-41a4-be34-2915cec92732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:50.338962 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2ee685-51d2-41a4-be34-2915cec92732 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:50.339448 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2ee685-51d2-41a4-be34-2915cec92732 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:50.340040 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2ee685-51d2-41a4-be34-2915cec92732 None admin] PATH_INFO: `/v3/projects/cc6e57482c554178aa9b830aa0b83a96` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:50.344184 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2ee685-51d2-41a4-be34-2915cec92732 None admin] RBAC: Authorizing `identity:get_project(project_id=cc6e57482c554178aa9b830aa0b83a96)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:50.345902 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2ee685-51d2-41a4-be34-2915cec92732 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:50.348321 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 95/198] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:23:50 2022] GET /identity/v3/projects/cc6e57482c554178aa9b830aa0b83a96 => generated 302 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:23:50.354306 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2ee685-51d2-41a4-be34-2915cec92732 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:50.368354 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:50.397165 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468663d0-90d9-4193-8666-e8c779d136fe None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:50.439221 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35d7d9b-4413-4761-bbfa-78b85fadabb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:50.439994 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35d7d9b-4413-4761-bbfa-78b85fadabb6 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:50.440197 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35d7d9b-4413-4761-bbfa-78b85fadabb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:50.440478 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35d7d9b-4413-4761-bbfa-78b85fadabb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:50.441429 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35d7d9b-4413-4761-bbfa-78b85fadabb6 None admin] RBAC: Adding query filter params (role.id=729cd9c0f42d47b6a8eee01e5f5a2d28, scope.project.id=cc6e57482c554178aa9b830aa0b83a96, user.id=993726e7f21a4c3eb6bb68bde8225406) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:50.441782 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35d7d9b-4413-4761-bbfa-78b85fadabb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:50.443292 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35d7d9b-4413-4761-bbfa-78b85fadabb6 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:50.449986 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 104/199] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 18 16:23:50 2022] GET /identity/v3/role_assignments?user.id=993726e7f21a4c3eb6bb68bde8225406&scope.project.id=cc6e57482c554178aa9b830aa0b83a96&role.id=729cd9c0f42d47b6a8eee01e5f5a2d28 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:50.532274 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:50.574515 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2ee685-51d2-41a4-be34-2915cec92732 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:50.613368 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9f7f14-74c6-4575-b28c-be3f1456deb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:50.613860 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f7f14-74c6-4575-b28c-be3f1456deb9 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:50.614149 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f7f14-74c6-4575-b28c-be3f1456deb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:50.614432 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f7f14-74c6-4575-b28c-be3f1456deb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:50.615290 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9f7f14-74c6-4575-b28c-be3f1456deb9 None admin] RBAC: Adding query filter params (role.id=98eaadb3669f4b608d572c001742f559, scope.project.id=cc6e57482c554178aa9b830aa0b83a96, user.id=993726e7f21a4c3eb6bb68bde8225406) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:50.615696 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9f7f14-74c6-4575-b28c-be3f1456deb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:50.617185 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9f7f14-74c6-4575-b28c-be3f1456deb9 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:50.624244 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 96/200] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 18 16:23:50 2022] GET /identity/v3/role_assignments?user.id=993726e7f21a4c3eb6bb68bde8225406&scope.project.id=cc6e57482c554178aa9b830aa0b83a96&role.id=98eaadb3669f4b608d572c001742f559 => generated 259 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:51.392599 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d956d6-156c-4d37-907a-71b73d08b722 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:51.393056 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d956d6-156c-4d37-907a-71b73d08b722 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:51.393341 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d956d6-156c-4d37-907a-71b73d08b722 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:51.395004 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 105/201] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:23:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:51.400390 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a965b2-7536-49e7-a4d6-395cf2f72f62 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:51.400683 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a965b2-7536-49e7-a4d6-395cf2f72f62 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:51.400966 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a965b2-7536-49e7-a4d6-395cf2f72f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:51.422119 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-03a965b2-7536-49e7-a4d6-395cf2f72f62 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:51.423314 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-03a965b2-7536-49e7-a4d6-395cf2f72f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:51.510138 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 97/202] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:51.553821 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d956d6-156c-4d37-907a-71b73d08b722 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:51.672500 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d956d6-156c-4d37-907a-71b73d08b722 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:51.707887 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0bcd30-28e7-4619-bf1a-33ea2a8a407d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:51.708684 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0bcd30-28e7-4619-bf1a-33ea2a8a407d None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:51.708973 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0bcd30-28e7-4619-bf1a-33ea2a8a407d None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:51.709250 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0bcd30-28e7-4619-bf1a-33ea2a8a407d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:51.721960 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0bcd30-28e7-4619-bf1a-33ea2a8a407d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:51.730613 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-cb0bcd30-28e7-4619-bf1a-33ea2a8a407d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:23:51.730613 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:51.730613 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:51.730613 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:51.730613 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:51.730613 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:51.730613 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:51.730613 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:51.730613 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:51.730613 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:51.730613 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:51.730613 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:51.730613 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:51.730613 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:51.730613 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:51.730613 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:51.731608 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:51.731608 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:51.731608 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:51.731608 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:51.731608 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:51.731608 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:51.731608 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:51.731608 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:51.731608 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:51.731608 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:51.731608 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:51.731608 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:51.731608 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:51.731608 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:51.731608 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:51.731608 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:51.732415 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:51.732415 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:51.732415 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:51.732415 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:51.732415 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:51.732415 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:51.732415 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:51.732415 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:51.732415 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:51.732415 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:51.732415 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:23:51.732415 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:23:51.732415 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:51.732415 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 106/203] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:23:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:23:51.739471 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a965b2-7536-49e7-a4d6-395cf2f72f62 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:51.834496 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:51.888666 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a965b2-7536-49e7-a4d6-395cf2f72f62 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:51.931196 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9f5886-a7db-4aeb-bc19-8203aae19e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:51.931838 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f5886-a7db-4aeb-bc19-8203aae19e99 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:51.932125 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f5886-a7db-4aeb-bc19-8203aae19e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:51.932404 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f5886-a7db-4aeb-bc19-8203aae19e99 None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:51.933180 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9f5886-a7db-4aeb-bc19-8203aae19e99 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:51.933485 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9f5886-a7db-4aeb-bc19-8203aae19e99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:51.934881 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9f5886-a7db-4aeb-bc19-8203aae19e99 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:51.944345 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 98/204] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 18 16:23:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:51.950469 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0bcd30-28e7-4619-bf1a-33ea2a8a407d None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:52.113947 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0bcd30-28e7-4619-bf1a-33ea2a8a407d None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:52.149791 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca9d0fc-4372-47d1-a571-052ff86922fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:52.150969 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca9d0fc-4372-47d1-a571-052ff86922fb None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:52.151254 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca9d0fc-4372-47d1-a571-052ff86922fb None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:52.151563 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca9d0fc-4372-47d1-a571-052ff86922fb None admin] PATH_INFO: `/v3/users/77e423c38dcf4d16a61998f6eb9e4ca3` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:52.154157 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca9d0fc-4372-47d1-a571-052ff86922fb None admin] RBAC: Authorizing `identity:get_user(user_id=77e423c38dcf4d16a61998f6eb9e4ca3)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:52.156428 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca9d0fc-4372-47d1-a571-052ff86922fb None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:52.158363 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 107/205] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:51 2022] GET /identity/v3/users/77e423c38dcf4d16a61998f6eb9e4ca3 => generated 289 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:52.164233 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9f5886-a7db-4aeb-bc19-8203aae19e99 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:52.285410 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9f5886-a7db-4aeb-bc19-8203aae19e99 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:52.321366 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:52.339523 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d247811-39ef-416f-aecf-cbffa917e18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:52.339523 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d247811-39ef-416f-aecf-cbffa917e18f None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:52.339523 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d247811-39ef-416f-aecf-cbffa917e18f None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:52.339523 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d247811-39ef-416f-aecf-cbffa917e18f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:52.400809 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b0083a-7148-443d-a86a-11695735b02b None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:52.401439 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b0083a-7148-443d-a86a-11695735b02b None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:52.401947 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b0083a-7148-443d-a86a-11695735b02b None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:52.403571 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 108/206] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:23:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:52.408928 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855806d0-cf36-44cf-9617-e76b32689c3a None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:52.409484 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855806d0-cf36-44cf-9617-e76b32689c3a None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:52.409963 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855806d0-cf36-44cf-9617-e76b32689c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:52.443082 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d247811-39ef-416f-aecf-cbffa917e18f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:52.445109 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d247811-39ef-416f-aecf-cbffa917e18f None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:52.456532 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-855806d0-cf36-44cf-9617-e76b32689c3a None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:52.457766 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-855806d0-cf36-44cf-9617-e76b32689c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:52.551924 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 109/207] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:23:52.552425 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:52.557766 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f9d18-dcb3-4763-8472-2768bdb9913c None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:52.558847 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f9d18-dcb3-4763-8472-2768bdb9913c None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:52.559362 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f9d18-dcb3-4763-8472-2768bdb9913c None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:52.560943 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 110/208] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:23:52 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:52.567839 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d808481a-4cf7-49aa-a02c-75dc98a02e48 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:52.568377 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d808481a-4cf7-49aa-a02c-75dc98a02e48 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:52.568828 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d808481a-4cf7-49aa-a02c-75dc98a02e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:52.570566 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 99/209] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:23:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:23:52.576410 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d247811-39ef-416f-aecf-cbffa917e18f None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:52.598860 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-d808481a-4cf7-49aa-a02c-75dc98a02e48 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:52.600567 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d808481a-4cf7-49aa-a02c-75dc98a02e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:52.615361 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:52.694967 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 111/210] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:23:52.696145 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d808481a-4cf7-49aa-a02c-75dc98a02e48 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:52.711934 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d247811-39ef-416f-aecf-cbffa917e18f None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:52.762881 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74abca42-bd27-42cc-a5bc-2b701c6997e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:52.763800 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74abca42-bd27-42cc-a5bc-2b701c6997e3 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:52.764075 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74abca42-bd27-42cc-a5bc-2b701c6997e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:52.764344 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74abca42-bd27-42cc-a5bc-2b701c6997e3 None admin] PATH_INFO: `/v3/projects/04afdb2deaea49078a14a588c9d6de7e` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:52.765559 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74abca42-bd27-42cc-a5bc-2b701c6997e3 None admin] RBAC: Authorizing `identity:get_project(project_id=04afdb2deaea49078a14a588c9d6de7e)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:52.767332 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74abca42-bd27-42cc-a5bc-2b701c6997e3 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:52.769122 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 100/211] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:23:52 2022] GET /identity/v3/projects/04afdb2deaea49078a14a588c9d6de7e => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:52.770084 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74abca42-bd27-42cc-a5bc-2b701c6997e3 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:52.826688 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d808481a-4cf7-49aa-a02c-75dc98a02e48 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:52.861833 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:52.872620 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27d562d-aeef-4725-aee5-a73c2ec3dfe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:52.874530 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d562d-aeef-4725-aee5-a73c2ec3dfe5 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:52.874920 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d562d-aeef-4725-aee5-a73c2ec3dfe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:52.875300 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d562d-aeef-4725-aee5-a73c2ec3dfe5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:52.876657 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27d562d-aeef-4725-aee5-a73c2ec3dfe5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:52.878226 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27d562d-aeef-4725-aee5-a73c2ec3dfe5 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:52.880773 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 112/212] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:23:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:52.882103 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27d562d-aeef-4725-aee5-a73c2ec3dfe5 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:52.904967 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:52.914085 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74abca42-bd27-42cc-a5bc-2b701c6997e3 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:52.969443 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362ce8a7-ea99-4299-a821-e5ee271d6a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:52.970524 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ce8a7-ea99-4299-a821-e5ee271d6a2c None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:52.970945 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ce8a7-ea99-4299-a821-e5ee271d6a2c None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:52.971353 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ce8a7-ea99-4299-a821-e5ee271d6a2c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:52.988796 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362ce8a7-ea99-4299-a821-e5ee271d6a2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:53.001213 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-362ce8a7-ea99-4299-a821-e5ee271d6a2c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:23:53.001213 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:53.001213 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:53.001213 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:53.001213 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:53.001213 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:53.001213 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:53.001213 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:53.001213 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:53.001213 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:53.001213 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:53.001213 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:53.001213 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:53.001213 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:53.001213 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:53.001213 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:53.002311 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:53.002311 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:53.002311 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:53.002311 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:53.002311 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:53.002311 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:53.002311 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:53.002311 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:53.002311 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:53.002311 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:53.002311 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:53.002311 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:53.002311 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:53.002311 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:53.002311 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:53.002311 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:53.003300 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:53.003300 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:53.003300 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:53.003300 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:53.003300 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:53.003300 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:53.003300 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:53.003300 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:53.003300 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:53.003300 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:53.003300 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:23:53.003300 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:23:53.005327 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27d562d-aeef-4725-aee5-a73c2ec3dfe5 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:53.005752 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 101/213] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:23:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:23:53.006762 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362ce8a7-ea99-4299-a821-e5ee271d6a2c None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:53.048589 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cddfa7-5524-42c1-9dd0-d02f09fdcfa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:53.049369 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cddfa7-5524-42c1-9dd0-d02f09fdcfa1 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:53.049727 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cddfa7-5524-42c1-9dd0-d02f09fdcfa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:53.050071 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cddfa7-5524-42c1-9dd0-d02f09fdcfa1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:53.050984 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cddfa7-5524-42c1-9dd0-d02f09fdcfa1 None admin] RBAC: Adding query filter params (role.id=2ba8d51abb6e4efa872c82ac8c9ba4a8, scope.project.id=04afdb2deaea49078a14a588c9d6de7e, user.id=77e423c38dcf4d16a61998f6eb9e4ca3) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:53.051366 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cddfa7-5524-42c1-9dd0-d02f09fdcfa1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:53.052927 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cddfa7-5524-42c1-9dd0-d02f09fdcfa1 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:53.067868 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 113/214] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 18 16:23:52 2022] GET /identity/v3/role_assignments?user.id=77e423c38dcf4d16a61998f6eb9e4ca3&scope.project.id=04afdb2deaea49078a14a588c9d6de7e&role.id=2ba8d51abb6e4efa872c82ac8c9ba4a8 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:53.068706 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cddfa7-5524-42c1-9dd0-d02f09fdcfa1 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:53.130484 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362ce8a7-ea99-4299-a821-e5ee271d6a2c None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:53.155589 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:53.175324 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7b67c3-5c87-42f7-9a01-3ac6e866222b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:53.176057 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b67c3-5c87-42f7-9a01-3ac6e866222b None admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:53.176340 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b67c3-5c87-42f7-9a01-3ac6e866222b None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:53.176617 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b67c3-5c87-42f7-9a01-3ac6e866222b None admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:53.177359 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7b67c3-5c87-42f7-9a01-3ac6e866222b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:53.178846 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7b67c3-5c87-42f7-9a01-3ac6e866222b None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:53.207479 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cddfa7-5524-42c1-9dd0-d02f09fdcfa1 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:53.249240 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d327abd-c756-4892-a2b1-507fa83817f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:53.249933 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d327abd-c756-4892-a2b1-507fa83817f2 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:53.250221 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d327abd-c756-4892-a2b1-507fa83817f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:53.250497 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d327abd-c756-4892-a2b1-507fa83817f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:53.251264 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d327abd-c756-4892-a2b1-507fa83817f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:53.251610 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d327abd-c756-4892-a2b1-507fa83817f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:53.253013 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d327abd-c756-4892-a2b1-507fa83817f2 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:53.262678 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 114/215] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 18 16:23:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:53.263862 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70998b27-f119-4385-80ed-f9fe5c7fc84f None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:53.264154 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70998b27-f119-4385-80ed-f9fe5c7fc84f None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:53.264437 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70998b27-f119-4385-80ed-f9fe5c7fc84f None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:53.265743 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 115/216] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:53.266849 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaa837c-3d4a-461e-935e-5ac0755751dd None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:53.267157 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaa837c-3d4a-461e-935e-5ac0755751dd None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:53.267401 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaa837c-3d4a-461e-935e-5ac0755751dd None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:53.269431 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 116/217] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:23:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:53.269553 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaa837c-3d4a-461e-935e-5ac0755751dd None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:53.288543 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 102/218] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 18 16:23:53 2022] POST /identity/v3/users => generated 297 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:23:53.293930 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f078b8-d7cd-41c8-9480-0ed087abadbf None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:53.294250 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f078b8-d7cd-41c8-9480-0ed087abadbf None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:53.294534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f078b8-d7cd-41c8-9480-0ed087abadbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:53.315924 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-72f078b8-d7cd-41c8-9480-0ed087abadbf None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:53.317383 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-72f078b8-d7cd-41c8-9480-0ed087abadbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:53.321646 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:53.411015 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 103/219] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:53.413023 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750ac41-f86a-4122-bd5a-9bb962d721d7 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:53.413482 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750ac41-f86a-4122-bd5a-9bb962d721d7 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:53.413660 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750ac41-f86a-4122-bd5a-9bb962d721d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:53.433191 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaa837c-3d4a-461e-935e-5ac0755751dd None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:53.433424 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-1750ac41-f86a-4122-bd5a-9bb962d721d7 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:53.434613 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-1750ac41-f86a-4122-bd5a-9bb962d721d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:53.456544 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:53.476887 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf2438f-3519-42a5-b101-8d1ef4c3f59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:53.477881 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf2438f-3519-42a5-b101-8d1ef4c3f59c None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:53.478551 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf2438f-3519-42a5-b101-8d1ef4c3f59c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:53.478551 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf2438f-3519-42a5-b101-8d1ef4c3f59c None admin] PATH_INFO: `/v3/users/e822ec50158c4c8794b8bcddd123e2fe` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:53.494793 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf2438f-3519-42a5-b101-8d1ef4c3f59c None admin] RBAC: Authorizing `identity:get_user(user_id=e822ec50158c4c8794b8bcddd123e2fe)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:53.496302 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf2438f-3519-42a5-b101-8d1ef4c3f59c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:53.498078 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 117/220] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:53 2022] GET /identity/v3/users/e822ec50158c4c8794b8bcddd123e2fe => generated 297 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:53.498935 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf2438f-3519-42a5-b101-8d1ef4c3f59c None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:53.517355 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 104/221] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:53.518272 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1750ac41-f86a-4122-bd5a-9bb962d721d7 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:53.604006 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:53.628286 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf2438f-3519-42a5-b101-8d1ef4c3f59c None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:53.645927 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1750ac41-f86a-4122-bd5a-9bb962d721d7 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:53.666550 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03fd338-d3d7-4d27-bf65-fa14e1f018c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:53.667331 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fd338-d3d7-4d27-bf65-fa14e1f018c5 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:53.667673 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fd338-d3d7-4d27-bf65-fa14e1f018c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:53.667950 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fd338-d3d7-4d27-bf65-fa14e1f018c5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:53.678677 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03fd338-d3d7-4d27-bf65-fa14e1f018c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:53.684491 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0022d21-07e3-4b4f-9eaa-5874abb3ba94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:53.685150 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0022d21-07e3-4b4f-9eaa-5874abb3ba94 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:53.685435 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0022d21-07e3-4b4f-9eaa-5874abb3ba94 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:53.685714 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0022d21-07e3-4b4f-9eaa-5874abb3ba94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:53.688001 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-a03fd338-d3d7-4d27-bf65-fa14e1f018c5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:23:53.688001 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:53.688001 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:53.688001 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:53.688001 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:53.688001 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:53.688001 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:53.688001 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:53.688001 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:53.688001 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:53.688001 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:53.688001 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:53.688001 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:53.688001 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:53.688001 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:53.688001 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:53.688858 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:53.688858 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:53.688858 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:53.688858 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:53.688858 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:53.688858 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:53.688858 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:53.688858 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:53.688858 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:53.688858 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:53.688858 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:53.688858 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:53.688858 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:53.688858 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:53.688858 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:53.688858 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:53.689658 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:53.689658 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:53.689658 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:53.689658 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:53.689658 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:53.689658 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:53.689658 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:53.689658 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:53.689658 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:53.689658 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:53.689658 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:23:53.689658 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:23:53.689658 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 118/222] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 18 16:23:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:23:53.690309 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03fd338-d3d7-4d27-bf65-fa14e1f018c5 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:53.733472 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:53.766599 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0022d21-07e3-4b4f-9eaa-5874abb3ba94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:53.768434 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0022d21-07e3-4b4f-9eaa-5874abb3ba94 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:53.798618 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03fd338-d3d7-4d27-bf65-fa14e1f018c5 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:53.833701 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0060eb62-caca-4a2e-ad2f-c9edc82c9d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:53.843518 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0060eb62-caca-4a2e-ad2f-c9edc82c9d60 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:53.843518 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0060eb62-caca-4a2e-ad2f-c9edc82c9d60 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:53.843518 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0060eb62-caca-4a2e-ad2f-c9edc82c9d60 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:53.848487 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0060eb62-caca-4a2e-ad2f-c9edc82c9d60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:53.857289 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-0060eb62-caca-4a2e-ad2f-c9edc82c9d60 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:23:53.857289 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:53.857289 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:53.857289 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:53.857289 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:53.857289 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:53.857289 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:53.857289 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:53.857289 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:53.857289 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:53.857289 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:53.857289 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:53.857289 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:53.857289 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:53.857289 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:53.857289 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:53.858201 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:53.858201 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:53.858201 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:53.858201 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:53.858201 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:53.858201 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:53.858201 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:53.858201 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:53.858201 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:53.858201 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:53.858201 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:53.858201 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:53.858201 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:53.858201 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:53.858201 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:53.858201 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:53.858978 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:53.858978 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:53.858978 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:53.858978 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:53.858978 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:53.858978 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:53.858978 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:53.858978 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:53.858978 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:53.858978 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:53.858978 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:23:53.858978 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:23:53.860458 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:53.863562 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 119/223] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 18 16:23:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:23:53.864535 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0060eb62-caca-4a2e-ad2f-c9edc82c9d60 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:53.880980 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 105/224] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:23:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:23:53.882050 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0022d21-07e3-4b4f-9eaa-5874abb3ba94 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:53.968504 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:53.973410 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0060eb62-caca-4a2e-ad2f-c9edc82c9d60 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:53.996700 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0022d21-07e3-4b4f-9eaa-5874abb3ba94 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:54.022595 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be89b57-90b7-47cc-9880-45af9683372f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:54.023859 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be89b57-90b7-47cc-9880-45af9683372f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:54.025279 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be89b57-90b7-47cc-9880-45af9683372f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:54.025769 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be89b57-90b7-47cc-9880-45af9683372f None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:54.026951 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be89b57-90b7-47cc-9880-45af9683372f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:54.027708 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be89b57-90b7-47cc-9880-45af9683372f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:54.030094 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be89b57-90b7-47cc-9880-45af9683372f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:54.036389 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8f1f20-2c20-40cb-b41f-7ad3e8867878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:54.037163 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f1f20-2c20-40cb-b41f-7ad3e8867878 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:54.037447 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f1f20-2c20-40cb-b41f-7ad3e8867878 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:54.037731 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f1f20-2c20-40cb-b41f-7ad3e8867878 None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:54.038491 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8f1f20-2c20-40cb-b41f-7ad3e8867878 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:54.038796 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8f1f20-2c20-40cb-b41f-7ad3e8867878 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:54.041098 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8f1f20-2c20-40cb-b41f-7ad3e8867878 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:54.043078 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 120/225] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:23:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:54.044630 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be89b57-90b7-47cc-9880-45af9683372f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:54.055005 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 106/226] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:23:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:54.056496 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8f1f20-2c20-40cb-b41f-7ad3e8867878 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:54.092567 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:54.171277 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8f1f20-2c20-40cb-b41f-7ad3e8867878 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:54.178265 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be89b57-90b7-47cc-9880-45af9683372f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:54.219460 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaab3bc0-2c7f-408e-8118-26e5b50b4ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:54.220315 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab3bc0-2c7f-408e-8118-26e5b50b4ce4 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:54.220603 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab3bc0-2c7f-408e-8118-26e5b50b4ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:54.220883 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab3bc0-2c7f-408e-8118-26e5b50b4ce4 None admin] PATH_INFO: `/v3/users/993726e7f21a4c3eb6bb68bde8225406` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:54.221996 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517dbde4-55b8-486d-9e94-73926db90d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:54.223055 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517dbde4-55b8-486d-9e94-73926db90d23 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:54.223342 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517dbde4-55b8-486d-9e94-73926db90d23 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:54.223645 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517dbde4-55b8-486d-9e94-73926db90d23 None admin] PATH_INFO: `/v3/projects/cc6e57482c554178aa9b830aa0b83a96` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:54.224881 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaab3bc0-2c7f-408e-8118-26e5b50b4ce4 None admin] RBAC: Authorizing `identity:get_user(user_id=993726e7f21a4c3eb6bb68bde8225406)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:54.226266 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517dbde4-55b8-486d-9e94-73926db90d23 None admin] RBAC: Authorizing `identity:get_project(project_id=cc6e57482c554178aa9b830aa0b83a96)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:54.228373 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517dbde4-55b8-486d-9e94-73926db90d23 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:54.229888 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaab3bc0-2c7f-408e-8118-26e5b50b4ce4 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:54.230110 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 121/227] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:23:54 2022] GET /identity/v3/projects/cc6e57482c554178aa9b830aa0b83a96 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:54.230944 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517dbde4-55b8-486d-9e94-73926db90d23 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:54.236546 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 107/228] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:54 2022] GET /identity/v3/users/993726e7f21a4c3eb6bb68bde8225406 => generated 283 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:54.239061 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaab3bc0-2c7f-408e-8118-26e5b50b4ce4 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:54.265559 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:54.344510 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517dbde4-55b8-486d-9e94-73926db90d23 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:54.383059 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:54.386942 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaab3bc0-2c7f-408e-8118-26e5b50b4ce4 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:54.388614 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d7b11d-8987-413f-9530-3512f7dfdc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:54.389715 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7b11d-8987-413f-9530-3512f7dfdc10 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:54.390193 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7b11d-8987-413f-9530-3512f7dfdc10 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:54.390745 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7b11d-8987-413f-9530-3512f7dfdc10 None admin] PATH_INFO: `/v3/users/993726e7f21a4c3eb6bb68bde8225406` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:54.392227 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d7b11d-8987-413f-9530-3512f7dfdc10 None admin] RBAC: Authorizing `identity:get_user(user_id=993726e7f21a4c3eb6bb68bde8225406)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:54.393840 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d7b11d-8987-413f-9530-3512f7dfdc10 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:54.397781 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 122/229] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:54 2022] GET /identity/v3/users/993726e7f21a4c3eb6bb68bde8225406 => generated 283 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:54.397781 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d7b11d-8987-413f-9530-3512f7dfdc10 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:54.434794 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b52168-dff5-4e96-a772-7e9338424871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:54.436442 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b52168-dff5-4e96-a772-7e9338424871 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:54.436906 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b52168-dff5-4e96-a772-7e9338424871 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:54.437337 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b52168-dff5-4e96-a772-7e9338424871 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:54.438493 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b52168-dff5-4e96-a772-7e9338424871 None admin] RBAC: Adding query filter params (role.id=aa0552e8ba924f5d9cb1740da7d4a4dc, scope.project.id=cc6e57482c554178aa9b830aa0b83a96, user.id=e822ec50158c4c8794b8bcddd123e2fe) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:54.439006 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b52168-dff5-4e96-a772-7e9338424871 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:54.440769 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b52168-dff5-4e96-a772-7e9338424871 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:54.455533 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 108/230] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 18 16:23:54 2022] GET /identity/v3/role_assignments?user.id=e822ec50158c4c8794b8bcddd123e2fe&scope.project.id=cc6e57482c554178aa9b830aa0b83a96&role.id=aa0552e8ba924f5d9cb1740da7d4a4dc => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:54.455533 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b52168-dff5-4e96-a772-7e9338424871 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:54.553049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:54.563790 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d7b11d-8987-413f-9530-3512f7dfdc10 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:54.639547 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4126d68a-1e56-4ada-a74f-88dab95d9dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:54.640439 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126d68a-1e56-4ada-a74f-88dab95d9dbd None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:54.640718 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126d68a-1e56-4ada-a74f-88dab95d9dbd None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:54.640989 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126d68a-1e56-4ada-a74f-88dab95d9dbd None admin] PATH_INFO: `/v3/projects/cc6e57482c554178aa9b830aa0b83a96` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:54.643634 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4126d68a-1e56-4ada-a74f-88dab95d9dbd None admin] RBAC: Authorizing `identity:get_project(project_id=cc6e57482c554178aa9b830aa0b83a96)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:54.645149 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4126d68a-1e56-4ada-a74f-88dab95d9dbd None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:54.648310 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 123/231] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:23:54 2022] GET /identity/v3/projects/cc6e57482c554178aa9b830aa0b83a96 => generated 302 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:54.655267 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4126d68a-1e56-4ada-a74f-88dab95d9dbd None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:54.663604 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b52168-dff5-4e96-a772-7e9338424871 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:54.721987 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df59d760-9bb0-40c9-aee0-7feef4e39ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:54.723026 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59d760-9bb0-40c9-aee0-7feef4e39ae7 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:54.723313 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59d760-9bb0-40c9-aee0-7feef4e39ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:54.723683 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59d760-9bb0-40c9-aee0-7feef4e39ae7 None admin] PATH_INFO: `/v3/projects/cc6e57482c554178aa9b830aa0b83a96` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:54.724948 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df59d760-9bb0-40c9-aee0-7feef4e39ae7 None admin] RBAC: Authorizing `identity:get_project(project_id=cc6e57482c554178aa9b830aa0b83a96)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:54.726473 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df59d760-9bb0-40c9-aee0-7feef4e39ae7 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:54.729157 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 109/232] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:23:54 2022] GET /identity/v3/projects/cc6e57482c554178aa9b830aa0b83a96 => generated 302 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:54.742005 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df59d760-9bb0-40c9-aee0-7feef4e39ae7 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:54.795093 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:54.835733 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4126d68a-1e56-4ada-a74f-88dab95d9dbd None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:54.874419 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee27687-2264-429a-82a5-ea7c13b2bfc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:54.876446 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee27687-2264-429a-82a5-ea7c13b2bfc3 None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:54.877964 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee27687-2264-429a-82a5-ea7c13b2bfc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:54.878395 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee27687-2264-429a-82a5-ea7c13b2bfc3 None admin] PATH_INFO: `/v3/projects/cc6e57482c554178aa9b830aa0b83a96/users/993726e7f21a4c3eb6bb68bde8225406/roles/729cd9c0f42d47b6a8eee01e5f5a2d28` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:54.884455 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee27687-2264-429a-82a5-ea7c13b2bfc3 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc6e57482c554178aa9b830aa0b83a96, user_id=993726e7f21a4c3eb6bb68bde8225406, role_id=729cd9c0f42d47b6a8eee01e5f5a2d28)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:54.886112 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee27687-2264-429a-82a5-ea7c13b2bfc3 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:54.926834 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df59d760-9bb0-40c9-aee0-7feef4e39ae7 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:54.969483 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 124/233] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 18 16:23:54 2022] PUT /identity/v3/projects/cc6e57482c554178aa9b830aa0b83a96/users/993726e7f21a4c3eb6bb68bde8225406/roles/729cd9c0f42d47b6a8eee01e5f5a2d28 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:23:54.972388 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b47f2e-c330-4d3e-96c6-0ff2856661fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:54.973569 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b47f2e-c330-4d3e-96c6-0ff2856661fc None admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:54.974071 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b47f2e-c330-4d3e-96c6-0ff2856661fc None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:54.974498 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b47f2e-c330-4d3e-96c6-0ff2856661fc None admin] PATH_INFO: `/v3/projects/cc6e57482c554178aa9b830aa0b83a96/users/993726e7f21a4c3eb6bb68bde8225406/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:54.977000 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b47f2e-c330-4d3e-96c6-0ff2856661fc None admin] RBAC: Authorizing `identity:create_grant(project_id=cc6e57482c554178aa9b830aa0b83a96, user_id=993726e7f21a4c3eb6bb68bde8225406, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:54.979584 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b47f2e-c330-4d3e-96c6-0ff2856661fc None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:55.052992 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 110/234] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 18 16:23:54 2022] PUT /identity/v3/projects/cc6e57482c554178aa9b830aa0b83a96/users/993726e7f21a4c3eb6bb68bde8225406/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:23:55.564427 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c04e715-45ce-471d-98dc-6816884e1552 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:55.565291 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c04e715-45ce-471d-98dc-6816884e1552 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:55.565776 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c04e715-45ce-471d-98dc-6816884e1552 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:55.567725 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 125/235] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:23:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:55.573732 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7321912-8d7a-4c63-a507-e99b367fe046 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:55.574301 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7321912-8d7a-4c63-a507-e99b367fe046 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:55.575184 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7321912-8d7a-4c63-a507-e99b367fe046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:55.601223 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-d7321912-8d7a-4c63-a507-e99b367fe046 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:55.602450 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-d7321912-8d7a-4c63-a507-e99b367fe046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:55.699440 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 111/236] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:55.735705 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c04e715-45ce-471d-98dc-6816884e1552 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:55.769305 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:55.885660 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c04e715-45ce-471d-98dc-6816884e1552 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:55.889768 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba3c94-50a5-4546-91d7-3b4229da84a1 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:55.890281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba3c94-50a5-4546-91d7-3b4229da84a1 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:55.890357 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba3c94-50a5-4546-91d7-3b4229da84a1 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:55.892165 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 112/237] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:23:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:55.898018 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a58708-c1f0-49c8-9307-bc2805a03e0e None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:55.898398 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a58708-c1f0-49c8-9307-bc2805a03e0e None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:55.898677 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a58708-c1f0-49c8-9307-bc2805a03e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:55.927918 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-25a58708-c1f0-49c8-9307-bc2805a03e0e None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:55.929128 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-25a58708-c1f0-49c8-9307-bc2805a03e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:55.943328 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc033cbb-1f38-42bf-b36e-4c5b22be6ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:55.944512 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc033cbb-1f38-42bf-b36e-4c5b22be6ec9 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:55.944991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc033cbb-1f38-42bf-b36e-4c5b22be6ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:55.945459 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc033cbb-1f38-42bf-b36e-4c5b22be6ec9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:55.957601 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc033cbb-1f38-42bf-b36e-4c5b22be6ec9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:55.966475 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-dc033cbb-1f38-42bf-b36e-4c5b22be6ec9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:23:55.966475 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:55.966475 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:55.966475 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:55.966475 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:55.966475 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:55.966475 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:55.966475 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:55.966475 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:55.966475 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:55.966475 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:55.966475 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:55.966475 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:55.966475 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:55.966475 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:55.966475 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:55.967382 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:55.967382 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:55.967382 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:55.967382 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:55.967382 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:55.967382 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:55.967382 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:55.967382 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:55.967382 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:55.967382 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:55.967382 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:55.967382 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:55.967382 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:55.967382 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:55.967382 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:55.967382 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:55.968201 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:55.968201 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:55.968201 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:55.968201 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:55.968201 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:55.968201 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:55.968201 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:55.968201 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:55.968201 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:55.968201 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:55.968201 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:23:55.968201 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:23:55.968785 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:55.970650 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 126/238] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:23:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:23:55.976512 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc033cbb-1f38-42bf-b36e-4c5b22be6ec9 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:56.084881 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:56.087606 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 113/239] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:23:56.103734 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc033cbb-1f38-42bf-b36e-4c5b22be6ec9 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:56.133269 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a58708-c1f0-49c8-9307-bc2805a03e0e None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:56.142267 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec91342-dd80-4bd5-88f6-75db55a3cde5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:56.143244 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec91342-dd80-4bd5-88f6-75db55a3cde5 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:56.143733 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec91342-dd80-4bd5-88f6-75db55a3cde5 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:56.144275 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec91342-dd80-4bd5-88f6-75db55a3cde5 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:56.145194 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec91342-dd80-4bd5-88f6-75db55a3cde5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:56.146075 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec91342-dd80-4bd5-88f6-75db55a3cde5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:56.147837 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec91342-dd80-4bd5-88f6-75db55a3cde5 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:56.160147 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 127/240] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 18 16:23:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:56.166238 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec91342-dd80-4bd5-88f6-75db55a3cde5 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:56.321107 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a58708-c1f0-49c8-9307-bc2805a03e0e None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:56.364818 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a448bc79-9355-42fb-b503-de06cc5ffe62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:56.366042 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448bc79-9355-42fb-b503-de06cc5ffe62 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:56.366519 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448bc79-9355-42fb-b503-de06cc5ffe62 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:56.366976 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448bc79-9355-42fb-b503-de06cc5ffe62 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:56.368440 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a448bc79-9355-42fb-b503-de06cc5ffe62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:56.371266 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a448bc79-9355-42fb-b503-de06cc5ffe62 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:56.373532 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 114/241] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:23:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:23:56.381143 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a448bc79-9355-42fb-b503-de06cc5ffe62 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:56.414226 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:56.421473 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec91342-dd80-4bd5-88f6-75db55a3cde5 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:56.500770 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b075e33-3a98-4dcb-9ac6-cfe295972be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:56.501657 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b075e33-3a98-4dcb-9ac6-cfe295972be0 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:56.501945 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b075e33-3a98-4dcb-9ac6-cfe295972be0 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:56.502220 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b075e33-3a98-4dcb-9ac6-cfe295972be0 None admin] PATH_INFO: `/v3/users/fa41d8cbfdd64a5080b4ced87f44bc86` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:56.528711 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b075e33-3a98-4dcb-9ac6-cfe295972be0 None admin] RBAC: Authorizing `identity:get_user(user_id=fa41d8cbfdd64a5080b4ced87f44bc86)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:56.530227 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b075e33-3a98-4dcb-9ac6-cfe295972be0 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:56.532056 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 128/242] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:56 2022] GET /identity/v3/users/fa41d8cbfdd64a5080b4ced87f44bc86 => generated 297 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:23:56.535896 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a448bc79-9355-42fb-b503-de06cc5ffe62 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:56.537483 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b075e33-3a98-4dcb-9ac6-cfe295972be0 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:56.543810 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:56.574990 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e634921-a5c7-4ac0-8c04-2f6c2b67583b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:56.575722 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e634921-a5c7-4ac0-8c04-2f6c2b67583b None admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:56.576007 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e634921-a5c7-4ac0-8c04-2f6c2b67583b None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:56.576287 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e634921-a5c7-4ac0-8c04-2f6c2b67583b None admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:56.577020 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e634921-a5c7-4ac0-8c04-2f6c2b67583b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:56.578498 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e634921-a5c7-4ac0-8c04-2f6c2b67583b None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:56.674559 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b075e33-3a98-4dcb-9ac6-cfe295972be0 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:56.715515 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2f4ebd-0703-4862-bba6-021609b20b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:56.716265 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2f4ebd-0703-4862-bba6-021609b20b8f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:56.716586 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2f4ebd-0703-4862-bba6-021609b20b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:56.716868 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2f4ebd-0703-4862-bba6-021609b20b8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:56.764023 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 115/243] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 18 16:23:56 2022] POST /identity/v3/users => generated 293 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:23:56.800139 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2f4ebd-0703-4862-bba6-021609b20b8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:56.801674 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2f4ebd-0703-4862-bba6-021609b20b8f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:56.805527 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:56.914443 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 129/244] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:23:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 16:23:56.921246 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e634921-a5c7-4ac0-8c04-2f6c2b67583b None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:57.026141 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accf2fb4-72a4-4766-9ca5-e6e5086ca2b6 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:57.026532 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accf2fb4-72a4-4766-9ca5-e6e5086ca2b6 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:57.026727 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accf2fb4-72a4-4766-9ca5-e6e5086ca2b6 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:57.028449 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 130/245] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:23:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:57.033670 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d9d366-fea4-4e82-94ea-5a2fc779107c None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:57.033964 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d9d366-fea4-4e82-94ea-5a2fc779107c None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:57.034253 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d9d366-fea4-4e82-94ea-5a2fc779107c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:57.040367 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:57.060604 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-d1d9d366-fea4-4e82-94ea-5a2fc779107c None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:57.062404 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d1d9d366-fea4-4e82-94ea-5a2fc779107c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:57.074114 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e634921-a5c7-4ac0-8c04-2f6c2b67583b None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:57.120619 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67352a9-8b69-4f1b-bd36-a58f68c31971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:57.121633 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67352a9-8b69-4f1b-bd36-a58f68c31971 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:57.121992 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67352a9-8b69-4f1b-bd36-a58f68c31971 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:57.122342 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67352a9-8b69-4f1b-bd36-a58f68c31971 None admin] PATH_INFO: `/v3/projects/cc6e57482c554178aa9b830aa0b83a96` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:57.123791 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67352a9-8b69-4f1b-bd36-a58f68c31971 None admin] RBAC: Authorizing `identity:get_project(project_id=cc6e57482c554178aa9b830aa0b83a96)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:57.125349 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67352a9-8b69-4f1b-bd36-a58f68c31971 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:57.127991 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 116/246] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:23:56 2022] GET /identity/v3/projects/cc6e57482c554178aa9b830aa0b83a96 => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:57.133925 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67352a9-8b69-4f1b-bd36-a58f68c31971 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:57.167229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 131/247] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:57.191272 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d9d366-fea4-4e82-94ea-5a2fc779107c None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:57.238931 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:57.267481 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67352a9-8b69-4f1b-bd36-a58f68c31971 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:57.317383 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6618b166-5580-4405-a5ae-a8d804c1d313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:57.318089 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6618b166-5580-4405-a5ae-a8d804c1d313 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:57.318380 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6618b166-5580-4405-a5ae-a8d804c1d313 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:57.318792 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6618b166-5580-4405-a5ae-a8d804c1d313 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:57.319943 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6618b166-5580-4405-a5ae-a8d804c1d313 None admin] RBAC: Adding query filter params (role.id=2ba8d51abb6e4efa872c82ac8c9ba4a8, scope.project.id=cc6e57482c554178aa9b830aa0b83a96, user.id=fa41d8cbfdd64a5080b4ced87f44bc86) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:57.320147 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6618b166-5580-4405-a5ae-a8d804c1d313 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:57.321843 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6618b166-5580-4405-a5ae-a8d804c1d313 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:57.330488 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d9d366-fea4-4e82-94ea-5a2fc779107c None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:57.335683 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 117/248] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 18 16:23:57 2022] GET /identity/v3/role_assignments?user.id=fa41d8cbfdd64a5080b4ced87f44bc86&scope.project.id=cc6e57482c554178aa9b830aa0b83a96&role.id=2ba8d51abb6e4efa872c82ac8c9ba4a8 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:57.372667 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829942cb-c62a-49dc-b7d9-e2ee5cf12266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:57.373855 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829942cb-c62a-49dc-b7d9-e2ee5cf12266 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:57.374293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829942cb-c62a-49dc-b7d9-e2ee5cf12266 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:57.374717 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829942cb-c62a-49dc-b7d9-e2ee5cf12266 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:57.387957 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829942cb-c62a-49dc-b7d9-e2ee5cf12266 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:57.396269 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-829942cb-c62a-49dc-b7d9-e2ee5cf12266 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:23:57.396269 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:57.396269 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:57.396269 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:57.396269 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:57.396269 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:57.396269 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:57.396269 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:57.396269 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:57.396269 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:57.396269 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:57.396269 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:57.396269 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:57.396269 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:57.396269 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:57.396269 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:57.397474 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:57.397474 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:57.397474 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:57.397474 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:57.397474 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:57.397474 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:57.397474 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:57.397474 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:57.397474 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:57.397474 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:57.397474 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:57.397474 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:57.397474 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:57.397474 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:57.397474 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:57.397474 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:57.398480 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:57.398480 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:57.398480 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:57.398480 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:57.398480 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:57.398480 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:57.398480 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:57.398480 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:57.398480 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:57.398480 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:57.398480 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:23:57.398480 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:23:57.401883 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 132/249] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:23:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:23:57.408883 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6618b166-5580-4405-a5ae-a8d804c1d313 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:57.423973 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:57.527872 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6322c741-83e2-49c4-b716-88a73654232c None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:57.528428 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6322c741-83e2-49c4-b716-88a73654232c None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:57.528856 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6322c741-83e2-49c4-b716-88a73654232c None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:57.530461 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 133/250] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:23:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:23:57.536413 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a87d54-2aff-4dce-b8f4-af5aa19f273a None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:57.537029 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a87d54-2aff-4dce-b8f4-af5aa19f273a None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:57.537514 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a87d54-2aff-4dce-b8f4-af5aa19f273a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:57.538207 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6618b166-5580-4405-a5ae-a8d804c1d313 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:57.561594 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-e7a87d54-2aff-4dce-b8f4-af5aa19f273a None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:57.563231 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e7a87d54-2aff-4dce-b8f4-af5aa19f273a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:57.592255 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637eeaa5-72b1-44dc-873a-7bae3b25a30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:57.592874 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637eeaa5-72b1-44dc-873a-7bae3b25a30d None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:57.593157 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637eeaa5-72b1-44dc-873a-7bae3b25a30d None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:57.593714 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637eeaa5-72b1-44dc-873a-7bae3b25a30d None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:57.594193 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637eeaa5-72b1-44dc-873a-7bae3b25a30d None admin] RBAC: Adding query filter params (name=member) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:57.594499 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637eeaa5-72b1-44dc-873a-7bae3b25a30d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:57.595974 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637eeaa5-72b1-44dc-873a-7bae3b25a30d None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:57.608180 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 118/251] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 18 16:23:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:57.615420 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637eeaa5-72b1-44dc-873a-7bae3b25a30d None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:57.701269 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 134/252] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:57.715857 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:57.736607 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a87d54-2aff-4dce-b8f4-af5aa19f273a None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:57.777266 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:57.780637 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637eeaa5-72b1-44dc-873a-7bae3b25a30d None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:57.888283 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a87d54-2aff-4dce-b8f4-af5aa19f273a None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:57.892981 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c77e62-5f93-48f8-8387-bc9116e147ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:57.893897 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c77e62-5f93-48f8-8387-bc9116e147ea None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:57.894185 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c77e62-5f93-48f8-8387-bc9116e147ea None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:57.894468 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c77e62-5f93-48f8-8387-bc9116e147ea None admin] PATH_INFO: `/v3/users/e822ec50158c4c8794b8bcddd123e2fe` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:57.895748 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c77e62-5f93-48f8-8387-bc9116e147ea None admin] RBAC: Authorizing `identity:get_user(user_id=e822ec50158c4c8794b8bcddd123e2fe)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:57.897223 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c77e62-5f93-48f8-8387-bc9116e147ea None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:57.899105 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 119/253] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:57 2022] GET /identity/v3/users/e822ec50158c4c8794b8bcddd123e2fe => generated 297 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:57.900304 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90725648-c9da-4f43-9009-93ca465f1a11 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:57.900598 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90725648-c9da-4f43-9009-93ca465f1a11 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:57.900880 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90725648-c9da-4f43-9009-93ca465f1a11 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:57.902197 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 120/254] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:23:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:23:57.921065 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e8318-0e53-420a-99f9-0c201376dd58 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:57.921405 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e8318-0e53-420a-99f9-0c201376dd58 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:57.921923 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e8318-0e53-420a-99f9-0c201376dd58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:57.937529 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0e0b4e-f04b-437d-93b5-c58853b346d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:57.938721 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e0b4e-f04b-437d-93b5-c58853b346d7 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:57.939225 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e0b4e-f04b-437d-93b5-c58853b346d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:57.939744 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e0b4e-f04b-437d-93b5-c58853b346d7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:57.946102 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-fe0e8318-0e53-420a-99f9-0c201376dd58 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:23:57.948616 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-fe0e8318-0e53-420a-99f9-0c201376dd58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:23:57.952259 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0e0b4e-f04b-437d-93b5-c58853b346d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:57.965052 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-cc0e0b4e-f04b-437d-93b5-c58853b346d7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:23:57.965052 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:57.965052 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:57.965052 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:57.965052 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:57.965052 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:57.965052 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:57.965052 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:57.965052 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:57.965052 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:57.965052 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:57.965052 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:57.965052 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:57.965052 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:57.965052 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:57.965052 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:57.965991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:57.965991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:57.965991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:57.965991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:57.965991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:57.965991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:57.965991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:57.965991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:57.965991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:57.965991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:57.965991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:57.965991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:57.965991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:57.965991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:57.965991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:57.965991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:57.966776 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:57.966776 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:57.966776 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:57.966776 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:57.966776 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:57.966776 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:57.966776 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:57.966776 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:57.966776 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:57.966776 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:57.966776 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:23:57.966776 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:23:57.975596 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 135/255] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 18 16:23:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:23:57.977033 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0e0b4e-f04b-437d-93b5-c58853b346d7 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:58.068534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 121/256] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:23:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:23:58.069409 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0e8318-0e53-420a-99f9-0c201376dd58 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:58.160079 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:58.166699 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0e0b4e-f04b-437d-93b5-c58853b346d7 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:58.205192 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dded9c69-40ce-4608-bffa-594b4c076048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:58.206069 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded9c69-40ce-4608-bffa-594b4c076048 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:58.206358 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded9c69-40ce-4608-bffa-594b4c076048 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:58.206644 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded9c69-40ce-4608-bffa-594b4c076048 None admin] PATH_INFO: `/v3/projects/cc6e57482c554178aa9b830aa0b83a96` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:58.208054 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dded9c69-40ce-4608-bffa-594b4c076048 None admin] RBAC: Authorizing `identity:get_project(project_id=cc6e57482c554178aa9b830aa0b83a96)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:58.209770 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dded9c69-40ce-4608-bffa-594b4c076048 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:58.211777 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 136/257] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:23:57 2022] GET /identity/v3/projects/cc6e57482c554178aa9b830aa0b83a96 => generated 302 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:58.213111 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dded9c69-40ce-4608-bffa-594b4c076048 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:58.332847 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dded9c69-40ce-4608-bffa-594b4c076048 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:58.345960 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:58.373352 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdeced0-2ce1-4dc2-840d-77fab46784ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:58.374204 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdeced0-2ce1-4dc2-840d-77fab46784ac None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:58.374488 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdeced0-2ce1-4dc2-840d-77fab46784ac None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:58.374765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdeced0-2ce1-4dc2-840d-77fab46784ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:58.386709 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdeced0-2ce1-4dc2-840d-77fab46784ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:23:58.396039 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-cbdeced0-2ce1-4dc2-840d-77fab46784ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:23:58.396039 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:23:58.396039 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:23:58.396039 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:23:58.396039 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:23:58.396039 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:23:58.396039 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:23:58.396039 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:23:58.396039 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:23:58.396039 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:23:58.396039 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:23:58.396039 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:23:58.396039 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:23:58.396039 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:23:58.396039 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:23:58.396039 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:23:58.396937 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:23:58.396937 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:23:58.396937 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:23:58.396937 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:23:58.396937 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:23:58.396937 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:23:58.396937 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:23:58.396937 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:23:58.396937 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:23:58.396937 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:23:58.396937 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:23:58.396937 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:23:58.396937 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:23:58.396937 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:23:58.396937 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:23:58.396937 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:23:58.397728 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:23:58.397728 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:23:58.397728 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:23:58.397728 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:23:58.397728 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:23:58.397728 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:23:58.397728 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:23:58.397728 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:23:58.397728 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:23:58.397728 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:23:58.397728 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:23:58.397728 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:23:58.398789 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 137/258] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 18 16:23:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:23:58.398789 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdeced0-2ce1-4dc2-840d-77fab46784ac None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:58.425066 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:58.464587 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0e8318-0e53-420a-99f9-0c201376dd58 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:58.528066 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810c7000-6db7-47ac-b8f5-37e84e07024d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:58.528750 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c7000-6db7-47ac-b8f5-37e84e07024d None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:58.529029 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c7000-6db7-47ac-b8f5-37e84e07024d None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:58.529303 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c7000-6db7-47ac-b8f5-37e84e07024d None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:58.530086 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810c7000-6db7-47ac-b8f5-37e84e07024d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:58.530356 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810c7000-6db7-47ac-b8f5-37e84e07024d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:58.531743 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810c7000-6db7-47ac-b8f5-37e84e07024d None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:58.544926 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 122/259] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:23:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 476 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:58.554016 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810c7000-6db7-47ac-b8f5-37e84e07024d None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:58.584914 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdeced0-2ce1-4dc2-840d-77fab46784ac None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:58.638734 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80ae015-b9d8-4bc7-b318-552e8dc90f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:58.639660 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ae015-b9d8-4bc7-b318-552e8dc90f9c None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:58.639943 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ae015-b9d8-4bc7-b318-552e8dc90f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:58.640272 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ae015-b9d8-4bc7-b318-552e8dc90f9c None admin] PATH_INFO: `/v3/projects/cc6e57482c554178aa9b830aa0b83a96/users/e822ec50158c4c8794b8bcddd123e2fe/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:58.645343 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80ae015-b9d8-4bc7-b318-552e8dc90f9c None admin] RBAC: Authorizing `identity:create_grant(project_id=cc6e57482c554178aa9b830aa0b83a96, user_id=e822ec50158c4c8794b8bcddd123e2fe, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:58.646807 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80ae015-b9d8-4bc7-b318-552e8dc90f9c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:58.738236 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 138/260] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 18 16:23:58 2022] PUT /identity/v3/projects/cc6e57482c554178aa9b830aa0b83a96/users/e822ec50158c4c8794b8bcddd123e2fe/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:23:58.739156 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80ae015-b9d8-4bc7-b318-552e8dc90f9c None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:58.744228 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810c7000-6db7-47ac-b8f5-37e84e07024d None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:58.786333 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:58.811542 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b0cf4-549c-4d55-8033-3837e10a8b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:58.811542 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b0cf4-549c-4d55-8033-3837e10a8b8e None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:58.811542 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b0cf4-549c-4d55-8033-3837e10a8b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:58.811542 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b0cf4-549c-4d55-8033-3837e10a8b8e None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:58.811959 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9b0cf4-549c-4d55-8033-3837e10a8b8e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:23:58.811959 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9b0cf4-549c-4d55-8033-3837e10a8b8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:58.812677 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9b0cf4-549c-4d55-8033-3837e10a8b8e None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:58.834565 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 123/261] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:23:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:58.848590 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b0cf4-549c-4d55-8033-3837e10a8b8e None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:58.891931 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:58.936922 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80ae015-b9d8-4bc7-b318-552e8dc90f9c None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:59.009443 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a7f05d-3bd9-47dd-a9cd-384fb5cab61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:59.010358 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7f05d-3bd9-47dd-a9cd-384fb5cab61e None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:59.010642 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7f05d-3bd9-47dd-a9cd-384fb5cab61e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:59.010920 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7f05d-3bd9-47dd-a9cd-384fb5cab61e None admin] PATH_INFO: `/v3/users/993726e7f21a4c3eb6bb68bde8225406` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:59.012238 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a7f05d-3bd9-47dd-a9cd-384fb5cab61e None admin] RBAC: Authorizing `identity:get_user(user_id=993726e7f21a4c3eb6bb68bde8225406)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:59.013697 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a7f05d-3bd9-47dd-a9cd-384fb5cab61e None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:59.015623 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 139/262] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:58 2022] GET /identity/v3/users/993726e7f21a4c3eb6bb68bde8225406 => generated 283 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:59.021986 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a7f05d-3bd9-47dd-a9cd-384fb5cab61e None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:59.106338 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b0cf4-549c-4d55-8033-3837e10a8b8e None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:59.175125 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a78aa5a-ada7-485d-8279-c8ef50ec037f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:59.176045 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a78aa5a-ada7-485d-8279-c8ef50ec037f None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:59.176326 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a78aa5a-ada7-485d-8279-c8ef50ec037f None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:59.176604 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a78aa5a-ada7-485d-8279-c8ef50ec037f None admin] PATH_INFO: `/v3/users/993726e7f21a4c3eb6bb68bde8225406` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:59.179114 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a78aa5a-ada7-485d-8279-c8ef50ec037f None admin] RBAC: Authorizing `identity:get_user(user_id=993726e7f21a4c3eb6bb68bde8225406)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:59.180605 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a78aa5a-ada7-485d-8279-c8ef50ec037f None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:59.182535 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 124/263] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 18 16:23:58 2022] GET /identity/v3/users/993726e7f21a4c3eb6bb68bde8225406 => generated 283 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:59.188511 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a78aa5a-ada7-485d-8279-c8ef50ec037f None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:59.205344 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a7f05d-3bd9-47dd-a9cd-384fb5cab61e None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:59.274168 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0393d66d-423c-4526-b797-2668804b7562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:59.276123 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0393d66d-423c-4526-b797-2668804b7562 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:59.276638 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0393d66d-423c-4526-b797-2668804b7562 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:59.277118 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0393d66d-423c-4526-b797-2668804b7562 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:59.295227 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:59.311891 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:59.386439 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a78aa5a-ada7-485d-8279-c8ef50ec037f None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:59.418689 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0393d66d-423c-4526-b797-2668804b7562 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:59.422698 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0393d66d-423c-4526-b797-2668804b7562 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:59.475607 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ed9264-a255-456f-b59c-40f930218fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:59.476286 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed9264-a255-456f-b59c-40f930218fc7 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:59.476571 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed9264-a255-456f-b59c-40f930218fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:59.476849 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed9264-a255-456f-b59c-40f930218fc7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:59.616520 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ed9264-a255-456f-b59c-40f930218fc7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:59.617914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ed9264-a255-456f-b59c-40f930218fc7 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:59.620726 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 140/264] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:23:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 16:23:59.627523 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0393d66d-423c-4526-b797-2668804b7562 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:59.718634 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:59.754184 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:23:59.830864 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0393d66d-423c-4526-b797-2668804b7562 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:23:59.842762 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 125/265] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:23:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 654 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:23:59.848710 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ed9264-a255-456f-b59c-40f930218fc7 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:23:59.899018 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5759ff-27a5-4265-8d1a-35645eca4302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:23:59.900540 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5759ff-27a5-4265-8d1a-35645eca4302 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:23:59.901821 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5759ff-27a5-4265-8d1a-35645eca4302 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:23:59.901821 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5759ff-27a5-4265-8d1a-35645eca4302 None admin] PATH_INFO: `/v3/projects/cc6e57482c554178aa9b830aa0b83a96` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:23:59.904657 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5759ff-27a5-4265-8d1a-35645eca4302 None admin] RBAC: Authorizing `identity:get_project(project_id=cc6e57482c554178aa9b830aa0b83a96)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:23:59.906946 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5759ff-27a5-4265-8d1a-35645eca4302 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:23:59.908968 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 141/266] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:23:59 2022] GET /identity/v3/projects/cc6e57482c554178aa9b830aa0b83a96 => generated 302 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:23:59.917004 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5759ff-27a5-4265-8d1a-35645eca4302 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:00.027310 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ed9264-a255-456f-b59c-40f930218fc7 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:00.070527 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b1881b-8a12-4208-b28f-8d22ad8dc7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:00.072621 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1881b-8a12-4208-b28f-8d22ad8dc7ab None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:00.073291 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1881b-8a12-4208-b28f-8d22ad8dc7ab None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:00.074322 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1881b-8a12-4208-b28f-8d22ad8dc7ab None admin] PATH_INFO: `/v3/projects/cc6e57482c554178aa9b830aa0b83a96` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:00.075863 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b1881b-8a12-4208-b28f-8d22ad8dc7ab None admin] RBAC: Authorizing `identity:get_project(project_id=cc6e57482c554178aa9b830aa0b83a96)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:00.078142 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b1881b-8a12-4208-b28f-8d22ad8dc7ab None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:00.080297 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 126/267] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:23:59 2022] GET /identity/v3/projects/cc6e57482c554178aa9b830aa0b83a96 => generated 302 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:00.081859 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08c193-83b5-45a1-92f5-0a2027e6fd4d None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:00.082350 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08c193-83b5-45a1-92f5-0a2027e6fd4d None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:00.082829 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08c193-83b5-45a1-92f5-0a2027e6fd4d None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:00.084728 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 127/268] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:00.086038 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9086e81-b483-4988-8515-3d66af05cd4d None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:00.090361 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5759ff-27a5-4265-8d1a-35645eca4302 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:00.090791 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9086e81-b483-4988-8515-3d66af05cd4d None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:00.091187 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9086e81-b483-4988-8515-3d66af05cd4d None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:00.092850 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 128/269] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:00.093913 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9086e81-b483-4988-8515-3d66af05cd4d None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:00.107950 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:00.125540 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:00.137783 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673b6c2f-f460-4a3d-8ca6-fe68616db3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:00.138725 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b6c2f-f460-4a3d-8ca6-fe68616db3f2 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:00.139010 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b6c2f-f460-4a3d-8ca6-fe68616db3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:00.139286 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b6c2f-f460-4a3d-8ca6-fe68616db3f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:00.140166 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673b6c2f-f460-4a3d-8ca6-fe68616db3f2 None admin] RBAC: Adding query filter params (role.id=729cd9c0f42d47b6a8eee01e5f5a2d28, scope.project.id=cc6e57482c554178aa9b830aa0b83a96, user.id=993726e7f21a4c3eb6bb68bde8225406) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:00.140478 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673b6c2f-f460-4a3d-8ca6-fe68616db3f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:00.141943 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673b6c2f-f460-4a3d-8ca6-fe68616db3f2 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:00.151344 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 142/270] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 18 16:23:59 2022] GET /identity/v3/role_assignments?user.id=993726e7f21a4c3eb6bb68bde8225406&scope.project.id=cc6e57482c554178aa9b830aa0b83a96&role.id=729cd9c0f42d47b6a8eee01e5f5a2d28 => generated 608 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:00.153117 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a39ec7a-9c3a-4433-b9ad-82095dc072d8 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:00.153676 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a39ec7a-9c3a-4433-b9ad-82095dc072d8 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:00.153976 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a39ec7a-9c3a-4433-b9ad-82095dc072d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:00.175780 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-0a39ec7a-9c3a-4433-b9ad-82095dc072d8 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:00.177062 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-0a39ec7a-9c3a-4433-b9ad-82095dc072d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:00.252775 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9086e81-b483-4988-8515-3d66af05cd4d None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:00.269141 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 143/271] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:00.270713 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b346d8d7-7ce3-445e-836b-33721ccfb064 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:00.270947 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b346d8d7-7ce3-445e-836b-33721ccfb064 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:00.271229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b346d8d7-7ce3-445e-836b-33721ccfb064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:00.298216 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a13f09b-9b41-4354-bdd0-42c677973a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:00.299332 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a13f09b-9b41-4354-bdd0-42c677973a0d None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:00.299896 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a13f09b-9b41-4354-bdd0-42c677973a0d None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:00.300424 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a13f09b-9b41-4354-bdd0-42c677973a0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:00.300694 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-b346d8d7-7ce3-445e-836b-33721ccfb064 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:00.301751 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a13f09b-9b41-4354-bdd0-42c677973a0d None admin] RBAC: Adding query filter params (role.id=98eaadb3669f4b608d572c001742f559, scope.project.id=cc6e57482c554178aa9b830aa0b83a96, user.id=993726e7f21a4c3eb6bb68bde8225406) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:00.301961 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-b346d8d7-7ce3-445e-836b-33721ccfb064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:00.302499 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a13f09b-9b41-4354-bdd0-42c677973a0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:00.304298 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a13f09b-9b41-4354-bdd0-42c677973a0d None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:00.312652 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 129/272] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 18 16:24:00 2022] GET /identity/v3/role_assignments?user.id=993726e7f21a4c3eb6bb68bde8225406&scope.project.id=cc6e57482c554178aa9b830aa0b83a96&role.id=98eaadb3669f4b608d572c001742f559 => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:00.314097 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d6e6ce-a6b2-49b7-84ad-1db19d38e26d None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:00.314395 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d6e6ce-a6b2-49b7-84ad-1db19d38e26d None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:00.314670 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d6e6ce-a6b2-49b7-84ad-1db19d38e26d None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:00.316341 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 130/273] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:00.322642 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98158cae-9dcb-448b-867f-3a8fdcbdb47a None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:00.322942 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98158cae-9dcb-448b-867f-3a8fdcbdb47a None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:00.323238 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98158cae-9dcb-448b-867f-3a8fdcbdb47a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:00.341268 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-98158cae-9dcb-448b-867f-3a8fdcbdb47a None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:00.342545 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-98158cae-9dcb-448b-867f-3a8fdcbdb47a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:00.409258 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 144/274] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:00.411027 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b346d8d7-7ce3-445e-836b-33721ccfb064 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:00.428265 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 131/275] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:00.448003 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:00.448956 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98158cae-9dcb-448b-867f-3a8fdcbdb47a None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:00.495884 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:00.535566 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b346d8d7-7ce3-445e-836b-33721ccfb064 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:00.572548 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98158cae-9dcb-448b-867f-3a8fdcbdb47a None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:00.582552 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b80ed6-18c4-4921-90ec-bc5f25ca8d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:00.584276 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b80ed6-18c4-4921-90ec-bc5f25ca8d51 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:00.584276 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b80ed6-18c4-4921-90ec-bc5f25ca8d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:00.584669 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b80ed6-18c4-4921-90ec-bc5f25ca8d51 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:00.586382 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b80ed6-18c4-4921-90ec-bc5f25ca8d51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:00.588811 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b80ed6-18c4-4921-90ec-bc5f25ca8d51 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:00.591084 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 145/276] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:00.591882 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b80ed6-18c4-4921-90ec-bc5f25ca8d51 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:00.623266 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875b91ca-837d-4298-8caa-2db497253f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:00.624676 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b91ca-837d-4298-8caa-2db497253f62 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:00.624993 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b91ca-837d-4298-8caa-2db497253f62 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:00.625501 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b91ca-837d-4298-8caa-2db497253f62 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:00.640459 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875b91ca-837d-4298-8caa-2db497253f62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:00.648688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-875b91ca-837d-4298-8caa-2db497253f62 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:24:00.648688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:00.648688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:00.648688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:00.648688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:00.648688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:00.648688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:00.648688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:00.648688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:00.648688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:00.648688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:00.648688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:00.648688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:00.648688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:00.648688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:00.648688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:00.650418 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:00.650418 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:00.650418 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:00.650418 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:00.650418 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:00.650418 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:00.650418 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:00.650418 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:00.650418 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:00.650418 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:00.650418 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:00.650418 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:00.650418 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:00.650418 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:00.650418 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:00.650418 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:00.657083 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:00.657083 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:00.657083 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:00.657083 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:00.657083 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:00.657083 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:00.657083 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:00.657083 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:00.657083 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:00.657083 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:00.657083 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:24:00.657083 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:00.657083 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 132/277] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:24:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:24:00.657083 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875b91ca-837d-4298-8caa-2db497253f62 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:00.726476 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:00.729978 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b80ed6-18c4-4921-90ec-bc5f25ca8d51 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:00.746409 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:00.766076 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875b91ca-837d-4298-8caa-2db497253f62 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:00.772450 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8843de77-4118-4f7f-b3aa-192f463cd43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:00.773236 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843de77-4118-4f7f-b3aa-192f463cd43f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:00.773520 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843de77-4118-4f7f-b3aa-192f463cd43f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:00.773794 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843de77-4118-4f7f-b3aa-192f463cd43f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:00.785411 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8843de77-4118-4f7f-b3aa-192f463cd43f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:00.794592 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-8843de77-4118-4f7f-b3aa-192f463cd43f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:24:00.794592 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:00.794592 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:00.794592 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:00.794592 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:00.794592 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:00.794592 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:00.794592 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:00.794592 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:00.794592 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:00.794592 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:00.794592 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:00.794592 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:00.794592 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:00.794592 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:00.794592 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:00.795676 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:00.795676 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:00.795676 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:00.795676 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:00.795676 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:00.795676 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:00.795676 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:00.795676 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:00.795676 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:00.795676 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:00.795676 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:00.795676 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:00.795676 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:00.795676 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:00.795676 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:00.795676 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:00.801439 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:00.801439 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:00.801439 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:00.801439 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:00.801439 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:00.801439 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:00.801439 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:00.801439 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:00.801439 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:00.801439 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:00.801439 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:24:00.801439 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:00.801439 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 146/278] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:24:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:00.801439 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8843de77-4118-4f7f-b3aa-192f463cd43f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:00.805037 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee4bbac-1a52-4550-8640-2d75b1fa9a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:00.805716 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4bbac-1a52-4550-8640-2d75b1fa9a45 None admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:00.806033 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4bbac-1a52-4550-8640-2d75b1fa9a45 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:00.806310 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4bbac-1a52-4550-8640-2d75b1fa9a45 None admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:00.807036 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee4bbac-1a52-4550-8640-2d75b1fa9a45 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:00.808515 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee4bbac-1a52-4550-8640-2d75b1fa9a45 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:00.905947 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8843de77-4118-4f7f-b3aa-192f463cd43f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:00.919461 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 133/279] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 18 16:24:00 2022] POST /identity/v3/users => generated 293 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:24:00.920442 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee4bbac-1a52-4550-8640-2d75b1fa9a45 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:00.958869 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983794f6-5b7a-4855-a19d-03388ac0cba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:00.960438 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983794f6-5b7a-4855-a19d-03388ac0cba5 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:00.961139 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983794f6-5b7a-4855-a19d-03388ac0cba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:00.961587 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983794f6-5b7a-4855-a19d-03388ac0cba5 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:00.963430 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983794f6-5b7a-4855-a19d-03388ac0cba5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:00.963918 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983794f6-5b7a-4855-a19d-03388ac0cba5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:00.966263 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983794f6-5b7a-4855-a19d-03388ac0cba5 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:00.978834 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 147/280] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 18 16:24:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:00.985302 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983794f6-5b7a-4855-a19d-03388ac0cba5 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:01.022335 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:01.062179 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee4bbac-1a52-4550-8640-2d75b1fa9a45 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:01.101505 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36456208-3f82-477e-85fa-fbd149c547a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:01.102438 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36456208-3f82-477e-85fa-fbd149c547a4 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:01.103567 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36456208-3f82-477e-85fa-fbd149c547a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:01.104053 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36456208-3f82-477e-85fa-fbd149c547a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:01.105021 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36456208-3f82-477e-85fa-fbd149c547a4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:01.105554 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36456208-3f82-477e-85fa-fbd149c547a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:01.107198 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36456208-3f82-477e-85fa-fbd149c547a4 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:01.113002 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:01.116308 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983794f6-5b7a-4855-a19d-03388ac0cba5 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:01.120535 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 134/281] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 18 16:24:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:01.128661 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36456208-3f82-477e-85fa-fbd149c547a4 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:01.164439 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e684547-c263-4425-afe2-8617c989ac6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:01.167756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e684547-c263-4425-afe2-8617c989ac6a None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:01.168101 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e684547-c263-4425-afe2-8617c989ac6a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:01.168381 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e684547-c263-4425-afe2-8617c989ac6a None admin] PATH_INFO: `/v3/users/fa41d8cbfdd64a5080b4ced87f44bc86` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:01.169708 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e684547-c263-4425-afe2-8617c989ac6a None admin] RBAC: Authorizing `identity:get_user(user_id=fa41d8cbfdd64a5080b4ced87f44bc86)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:01.171799 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e684547-c263-4425-afe2-8617c989ac6a None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:01.173684 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 148/282] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 18 16:24:00 2022] GET /identity/v3/users/fa41d8cbfdd64a5080b4ced87f44bc86 => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:01.180495 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e684547-c263-4425-afe2-8617c989ac6a None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:01.270328 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36456208-3f82-477e-85fa-fbd149c547a4 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:01.317192 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:01.320624 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6232f5df-3078-4809-9ec8-833eb38831be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:01.322212 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232f5df-3078-4809-9ec8-833eb38831be None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:01.322511 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232f5df-3078-4809-9ec8-833eb38831be None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:01.322790 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232f5df-3078-4809-9ec8-833eb38831be None admin] PATH_INFO: `/v3/users/02cd09c9ba5341508f7ef7e7e6c96f8a` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:01.333098 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e684547-c263-4425-afe2-8617c989ac6a None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:01.340428 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6232f5df-3078-4809-9ec8-833eb38831be None admin] RBAC: Authorizing `identity:get_user(user_id=02cd09c9ba5341508f7ef7e7e6c96f8a)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:01.341556 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6232f5df-3078-4809-9ec8-833eb38831be None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:01.347905 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 135/283] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 18 16:24:01 2022] GET /identity/v3/users/02cd09c9ba5341508f7ef7e7e6c96f8a => generated 293 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:01.350377 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6232f5df-3078-4809-9ec8-833eb38831be None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:01.382420 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b571e0-7df9-4cec-9b6e-4bb2214b644c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:01.383300 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b571e0-7df9-4cec-9b6e-4bb2214b644c None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:01.387432 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b571e0-7df9-4cec-9b6e-4bb2214b644c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:01.387934 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b571e0-7df9-4cec-9b6e-4bb2214b644c None admin] PATH_INFO: `/v3/projects/cc6e57482c554178aa9b830aa0b83a96` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:01.389393 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b571e0-7df9-4cec-9b6e-4bb2214b644c None admin] RBAC: Authorizing `identity:get_project(project_id=cc6e57482c554178aa9b830aa0b83a96)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:01.391063 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b571e0-7df9-4cec-9b6e-4bb2214b644c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:01.393455 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 149/284] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:24:01 2022] GET /identity/v3/projects/cc6e57482c554178aa9b830aa0b83a96 => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:01.402371 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b571e0-7df9-4cec-9b6e-4bb2214b644c None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:01.451889 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:01.531796 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6232f5df-3078-4809-9ec8-833eb38831be None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:01.582132 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b571e0-7df9-4cec-9b6e-4bb2214b644c None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:01.621596 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503924d3-2484-4364-80a3-3a37b6e957a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:01.623681 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503924d3-2484-4364-80a3-3a37b6e957a1 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:01.624456 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503924d3-2484-4364-80a3-3a37b6e957a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:01.625849 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503924d3-2484-4364-80a3-3a37b6e957a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:01.626900 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7775c5aa-f343-4fe3-bb5c-18e54dc2b647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:01.628142 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7775c5aa-f343-4fe3-bb5c-18e54dc2b647 None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:01.628686 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503924d3-2484-4364-80a3-3a37b6e957a1 None admin] RBAC: Adding query filter params (role.id=aa0552e8ba924f5d9cb1740da7d4a4dc, user.id=02cd09c9ba5341508f7ef7e7e6c96f8a, scope.system=all) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:01.629183 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503924d3-2484-4364-80a3-3a37b6e957a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:01.630837 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503924d3-2484-4364-80a3-3a37b6e957a1 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:01.631898 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7775c5aa-f343-4fe3-bb5c-18e54dc2b647 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:01.632414 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7775c5aa-f343-4fe3-bb5c-18e54dc2b647 None admin] PATH_INFO: `/v3/projects/cc6e57482c554178aa9b830aa0b83a96/users/fa41d8cbfdd64a5080b4ced87f44bc86/roles/2ba8d51abb6e4efa872c82ac8c9ba4a8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:01.634886 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7775c5aa-f343-4fe3-bb5c-18e54dc2b647 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc6e57482c554178aa9b830aa0b83a96, user_id=fa41d8cbfdd64a5080b4ced87f44bc86, role_id=2ba8d51abb6e4efa872c82ac8c9ba4a8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:01.636717 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7775c5aa-f343-4fe3-bb5c-18e54dc2b647 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:01.637716 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 136/285] 10.222.0.26 () {56 vars in 1308 bytes} [Mon Jul 18 16:24:01 2022] GET /identity/v3/role_assignments?user.id=02cd09c9ba5341508f7ef7e7e6c96f8a&scope.system=all&role.id=aa0552e8ba924f5d9cb1740da7d4a4dc => generated 226 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:01.644169 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d22221-0451-4520-ae8c-631dd0c55da5 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:01.644471 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d22221-0451-4520-ae8c-631dd0c55da5 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:01.644756 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d22221-0451-4520-ae8c-631dd0c55da5 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:01.646400 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 137/286] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:01 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:01.652898 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b9240-c965-4a1b-b96f-0a85bdfe9196 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:01.653691 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b9240-c965-4a1b-b96f-0a85bdfe9196 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:01.661017 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b9240-c965-4a1b-b96f-0a85bdfe9196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:01.675059 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-4e8b9240-c965-4a1b-b96f-0a85bdfe9196 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:01.676720 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-4e8b9240-c965-4a1b-b96f-0a85bdfe9196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:01.724028 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 150/287] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 18 16:24:01 2022] PUT /identity/v3/projects/cc6e57482c554178aa9b830aa0b83a96/users/fa41d8cbfdd64a5080b4ced87f44bc86/roles/2ba8d51abb6e4efa872c82ac8c9ba4a8 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:24:01.777865 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 138/288] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:24:01.817397 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7775c5aa-f343-4fe3-bb5c-18e54dc2b647 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:01.847756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:01.950355 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7775c5aa-f343-4fe3-bb5c-18e54dc2b647 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:01.990859 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edbab90-fe5c-40da-b639-16cbe4b035c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:01.991700 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edbab90-fe5c-40da-b639-16cbe4b035c1 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:01.991993 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edbab90-fe5c-40da-b639-16cbe4b035c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:01.992271 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edbab90-fe5c-40da-b639-16cbe4b035c1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:02.005004 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edbab90-fe5c-40da-b639-16cbe4b035c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:02.013756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-0edbab90-fe5c-40da-b639-16cbe4b035c1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:24:02.013756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:02.013756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:02.013756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:02.013756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:02.013756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:02.013756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:02.013756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:02.013756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:02.013756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:02.013756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:02.013756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:02.013756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:02.013756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:02.013756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:02.013756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:02.014680 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:02.014680 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:02.014680 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:02.014680 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:02.014680 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:02.014680 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:02.014680 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:02.014680 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:02.014680 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:02.014680 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:02.014680 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:02.014680 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:02.014680 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:02.014680 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:02.014680 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:02.014680 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:02.015626 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:02.015626 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:02.015626 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:02.015626 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:02.015626 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:02.015626 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:02.015626 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:02.015626 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:02.015626 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:02.015626 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:02.015626 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:24:02.015626 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:02.015626 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 151/289] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:24:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:02.021799 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8b9240-c965-4a1b-b96f-0a85bdfe9196 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:02.107633 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:02.179834 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8b9240-c965-4a1b-b96f-0a85bdfe9196 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:02.233386 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d68df2-1bdd-4e37-979f-2ca2b188186f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:02.234103 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d68df2-1bdd-4e37-979f-2ca2b188186f None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:02.234389 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d68df2-1bdd-4e37-979f-2ca2b188186f None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:02.234662 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d68df2-1bdd-4e37-979f-2ca2b188186f None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:02.235452 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d68df2-1bdd-4e37-979f-2ca2b188186f None admin] RBAC: Adding query filter params (name=member) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:02.235784 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d68df2-1bdd-4e37-979f-2ca2b188186f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:02.237210 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d68df2-1bdd-4e37-979f-2ca2b188186f None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:02.251212 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 139/290] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 18 16:24:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:02.256887 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edbab90-fe5c-40da-b639-16cbe4b035c1 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:02.411413 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:02.439038 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edbab90-fe5c-40da-b639-16cbe4b035c1 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:02.492899 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b7003b-260a-4826-960f-95fe57af1450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:02.493825 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b7003b-260a-4826-960f-95fe57af1450 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:02.494113 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b7003b-260a-4826-960f-95fe57af1450 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:02.494394 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b7003b-260a-4826-960f-95fe57af1450 None admin] PATH_INFO: `/v3/users/e822ec50158c4c8794b8bcddd123e2fe` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:02.495692 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b7003b-260a-4826-960f-95fe57af1450 None admin] RBAC: Authorizing `identity:get_user(user_id=e822ec50158c4c8794b8bcddd123e2fe)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:02.497182 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b7003b-260a-4826-960f-95fe57af1450 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:02.499235 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 152/291] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 18 16:24:02 2022] GET /identity/v3/users/e822ec50158c4c8794b8bcddd123e2fe => generated 297 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:02.504541 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d68df2-1bdd-4e37-979f-2ca2b188186f None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:02.670868 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d68df2-1bdd-4e37-979f-2ca2b188186f None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:02.688616 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:02.717992 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4162abdb-3889-4809-8e35-d10e3695fb37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:02.718666 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162abdb-3889-4809-8e35-d10e3695fb37 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:02.718953 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162abdb-3889-4809-8e35-d10e3695fb37 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:02.719234 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162abdb-3889-4809-8e35-d10e3695fb37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:02.815002 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4162abdb-3889-4809-8e35-d10e3695fb37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:02.816520 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4162abdb-3889-4809-8e35-d10e3695fb37 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:02.972533 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 140/292] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:24:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:24:02.977691 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b7003b-260a-4826-960f-95fe57af1450 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:03.149299 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b7003b-260a-4826-960f-95fe57af1450 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:03.197268 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec4f343-7d74-4070-8e76-bd9d1d97a407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:03.198449 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4f343-7d74-4070-8e76-bd9d1d97a407 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:03.198739 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4f343-7d74-4070-8e76-bd9d1d97a407 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:03.199021 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4f343-7d74-4070-8e76-bd9d1d97a407 None admin] PATH_INFO: `/v3/projects/cc6e57482c554178aa9b830aa0b83a96` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:03.200371 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec4f343-7d74-4070-8e76-bd9d1d97a407 None admin] RBAC: Authorizing `identity:get_project(project_id=cc6e57482c554178aa9b830aa0b83a96)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:03.201866 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec4f343-7d74-4070-8e76-bd9d1d97a407 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:03.203995 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 153/293] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:24:02 2022] GET /identity/v3/projects/cc6e57482c554178aa9b830aa0b83a96 => generated 302 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:03.209915 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4162abdb-3889-4809-8e35-d10e3695fb37 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:03.237055 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:03.365759 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4162abdb-3889-4809-8e35-d10e3695fb37 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:03.416667 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1677d601-af57-4551-999f-d9f48781d9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:03.417487 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1677d601-af57-4551-999f-d9f48781d9a3 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:03.417783 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1677d601-af57-4551-999f-d9f48781d9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:03.418067 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1677d601-af57-4551-999f-d9f48781d9a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:03.418950 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1677d601-af57-4551-999f-d9f48781d9a3 None admin] RBAC: Adding query filter params (role.id=aa0552e8ba924f5d9cb1740da7d4a4dc, scope.project.id=cc6e57482c554178aa9b830aa0b83a96, user.id=e822ec50158c4c8794b8bcddd123e2fe) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:03.419273 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1677d601-af57-4551-999f-d9f48781d9a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:03.420791 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1677d601-af57-4551-999f-d9f48781d9a3 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:03.428080 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 141/294] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 18 16:24:03 2022] GET /identity/v3/role_assignments?user.id=e822ec50158c4c8794b8bcddd123e2fe&scope.project.id=cc6e57482c554178aa9b830aa0b83a96&role.id=aa0552e8ba924f5d9cb1740da7d4a4dc => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:03.577314 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96f45d-8f74-4b43-a2d3-6c2cbea0e7c9 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:03.577649 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96f45d-8f74-4b43-a2d3-6c2cbea0e7c9 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:03.577977 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96f45d-8f74-4b43-a2d3-6c2cbea0e7c9 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:03.581162 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 154/295] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:03.585105 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f76008-e5ea-4b89-802b-5814997a38c4 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:03.585401 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f76008-e5ea-4b89-802b-5814997a38c4 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:03.585688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f76008-e5ea-4b89-802b-5814997a38c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:03.614280 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-67f76008-e5ea-4b89-802b-5814997a38c4 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:03.615601 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-67f76008-e5ea-4b89-802b-5814997a38c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:03.707338 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 142/296] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:24:03.735556 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa96f45d-8f74-4b43-a2d3-6c2cbea0e7c9 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:03.764466 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:03.781098 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32e239-990a-4d7a-a61f-a619399a71f9 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:03.781426 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32e239-990a-4d7a-a61f-a619399a71f9 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:03.781665 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32e239-990a-4d7a-a61f-a619399a71f9 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:03.783252 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 143/297] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:03.790170 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff6396-c697-4c3c-bc5b-e8cd4c3a0d92 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:03.790825 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff6396-c697-4c3c-bc5b-e8cd4c3a0d92 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:03.791332 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff6396-c697-4c3c-bc5b-e8cd4c3a0d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:03.810139 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-e2ff6396-c697-4c3c-bc5b-e8cd4c3a0d92 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:03.811707 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-e2ff6396-c697-4c3c-bc5b-e8cd4c3a0d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:03.824481 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:03.873017 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa96f45d-8f74-4b43-a2d3-6c2cbea0e7c9 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:03.887967 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 144/298] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:24:03.910693 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463cc8d8-71c5-4798-aa21-29ede8751791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:03.911515 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463cc8d8-71c5-4798-aa21-29ede8751791 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:03.911820 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463cc8d8-71c5-4798-aa21-29ede8751791 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:03.912099 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463cc8d8-71c5-4798-aa21-29ede8751791 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:03.914458 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff6396-c697-4c3c-bc5b-e8cd4c3a0d92 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:03.924811 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463cc8d8-71c5-4798-aa21-29ede8751791 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:03.933011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-463cc8d8-71c5-4798-aa21-29ede8751791 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:24:03.933011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:03.933011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:03.933011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:03.933011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:03.933011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:03.933011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:03.933011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:03.933011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:03.933011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:03.933011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:03.933011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:03.933011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:03.933011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:03.933011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:03.933011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:03.933944 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:03.933944 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:03.933944 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:03.933944 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:03.933944 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:03.933944 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:03.933944 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:03.933944 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:03.933944 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:03.933944 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:03.933944 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:03.933944 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:03.933944 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:03.933944 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:03.933944 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:03.933944 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:03.934729 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:03.934729 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:03.934729 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:03.934729 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:03.934729 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:03.934729 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:03.934729 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:03.934729 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:03.934729 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:03.934729 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:03.934729 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:24:03.934729 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:03.934729 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 155/299] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:24:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:03.940792 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463cc8d8-71c5-4798-aa21-29ede8751791 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:04.033747 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff6396-c697-4c3c-bc5b-e8cd4c3a0d92 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:04.047475 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:04.076022 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea395bf1-46b8-4a81-9894-70f6c9e16b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:04.077234 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea395bf1-46b8-4a81-9894-70f6c9e16b45 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:04.077665 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea395bf1-46b8-4a81-9894-70f6c9e16b45 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:04.078110 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea395bf1-46b8-4a81-9894-70f6c9e16b45 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:04.079707 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea395bf1-46b8-4a81-9894-70f6c9e16b45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:04.080558 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463cc8d8-71c5-4798-aa21-29ede8751791 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:04.081818 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea395bf1-46b8-4a81-9894-70f6c9e16b45 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:04.089476 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 145/300] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:04.090609 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea395bf1-46b8-4a81-9894-70f6c9e16b45 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:04.100786 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:04.125057 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581dd2fe-57dc-4c0b-9375-d304cb6a5045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:04.126160 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581dd2fe-57dc-4c0b-9375-d304cb6a5045 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:04.126544 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581dd2fe-57dc-4c0b-9375-d304cb6a5045 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:04.126947 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581dd2fe-57dc-4c0b-9375-d304cb6a5045 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:04.127961 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581dd2fe-57dc-4c0b-9375-d304cb6a5045 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:04.128681 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581dd2fe-57dc-4c0b-9375-d304cb6a5045 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:04.130437 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581dd2fe-57dc-4c0b-9375-d304cb6a5045 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:04.141454 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 156/301] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 18 16:24:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:04.147035 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581dd2fe-57dc-4c0b-9375-d304cb6a5045 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:04.213482 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea395bf1-46b8-4a81-9894-70f6c9e16b45 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:04.254019 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be9cbfd-dadc-422b-ad5e-56bee45182f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:04.255024 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be9cbfd-dadc-422b-ad5e-56bee45182f4 None admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:04.255315 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be9cbfd-dadc-422b-ad5e-56bee45182f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:04.255631 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be9cbfd-dadc-422b-ad5e-56bee45182f4 None admin] PATH_INFO: `/v3/groups` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:04.256567 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be9cbfd-dadc-422b-ad5e-56bee45182f4 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:04.258867 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be9cbfd-dadc-422b-ad5e-56bee45182f4 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:04.263592 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581dd2fe-57dc-4c0b-9375-d304cb6a5045 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:04.304546 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13b0ac8-e7d8-4f73-b4ad-b2503cea5313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:04.305428 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b0ac8-e7d8-4f73-b4ad-b2503cea5313 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:04.305712 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b0ac8-e7d8-4f73-b4ad-b2503cea5313 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:04.306061 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b0ac8-e7d8-4f73-b4ad-b2503cea5313 None admin] PATH_INFO: `/v3/users/cd8962c375294db8a7b38e35e75bbb52` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:04.310215 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:04.321436 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13b0ac8-e7d8-4f73-b4ad-b2503cea5313 None admin] RBAC: Authorizing `identity:get_user(user_id=cd8962c375294db8a7b38e35e75bbb52)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:04.323012 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13b0ac8-e7d8-4f73-b4ad-b2503cea5313 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:04.325022 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 157/302] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 18 16:24:04 2022] GET /identity/v3/users/cd8962c375294db8a7b38e35e75bbb52 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:04.330765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13b0ac8-e7d8-4f73-b4ad-b2503cea5313 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:04.332330 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 146/303] 10.222.0.26 () {60 vars in 1134 bytes} [Mon Jul 18 16:24:04 2022] POST /identity/v3/groups => generated 229 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:24:04.398876 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11641e52-c24d-40ea-a7d5-dd9e829f512b None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:04.399576 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11641e52-c24d-40ea-a7d5-dd9e829f512b None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:04.400648 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11641e52-c24d-40ea-a7d5-dd9e829f512b None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:04.402621 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 147/304] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:24:04.408763 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27322f48-bb7d-4086-a3e1-af82cadc6246 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:04.409405 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27322f48-bb7d-4086-a3e1-af82cadc6246 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:04.409910 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27322f48-bb7d-4086-a3e1-af82cadc6246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:04.429669 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-27322f48-bb7d-4086-a3e1-af82cadc6246 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:04.431276 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-27322f48-bb7d-4086-a3e1-af82cadc6246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:04.450770 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13b0ac8-e7d8-4f73-b4ad-b2503cea5313 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:04.494527 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c11858-89e7-4a84-a061-ff801acc71b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:04.495159 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c11858-89e7-4a84-a061-ff801acc71b1 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:04.495443 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c11858-89e7-4a84-a061-ff801acc71b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:04.495751 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c11858-89e7-4a84-a061-ff801acc71b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:04.497121 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c11858-89e7-4a84-a061-ff801acc71b1 None admin] RBAC: Adding query filter params (role.id=2ba8d51abb6e4efa872c82ac8c9ba4a8, user.id=cd8962c375294db8a7b38e35e75bbb52, scope.system=all) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:04.497441 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c11858-89e7-4a84-a061-ff801acc71b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:04.498896 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c11858-89e7-4a84-a061-ff801acc71b1 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:04.507082 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:04.507894 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 158/305] 10.222.0.26 () {56 vars in 1308 bytes} [Mon Jul 18 16:24:04 2022] GET /identity/v3/role_assignments?user.id=cd8962c375294db8a7b38e35e75bbb52&scope.system=all&role.id=2ba8d51abb6e4efa872c82ac8c9ba4a8 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:04.517648 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 148/306] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:04.544649 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c11858-89e7-4a84-a061-ff801acc71b1 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:04.583815 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3884a1f-361e-4715-87c1-38c89c7d1dac None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:04.584129 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3884a1f-361e-4715-87c1-38c89c7d1dac None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:04.584373 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3884a1f-361e-4715-87c1-38c89c7d1dac None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:04.586238 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 149/307] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:04.592812 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce754932-dbde-4c6a-8da9-5f56ca8053ef None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:04.593095 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce754932-dbde-4c6a-8da9-5f56ca8053ef None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:04.593370 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce754932-dbde-4c6a-8da9-5f56ca8053ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:04.614215 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-ce754932-dbde-4c6a-8da9-5f56ca8053ef None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:04.615922 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-ce754932-dbde-4c6a-8da9-5f56ca8053ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:04.620818 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:04.672316 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c11858-89e7-4a84-a061-ff801acc71b1 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:04.711794 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 150/308] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:04.724795 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1495f96d-90e5-43ad-8b1f-5e07561bd861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:04.725926 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1495f96d-90e5-43ad-8b1f-5e07561bd861 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:04.726406 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1495f96d-90e5-43ad-8b1f-5e07561bd861 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:04.726913 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1495f96d-90e5-43ad-8b1f-5e07561bd861 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:04.735485 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce754932-dbde-4c6a-8da9-5f56ca8053ef None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:04.740461 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1495f96d-90e5-43ad-8b1f-5e07561bd861 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:04.751480 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-1495f96d-90e5-43ad-8b1f-5e07561bd861 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:24:04.751480 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:04.751480 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:04.751480 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:04.751480 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:04.751480 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:04.751480 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:04.751480 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:04.751480 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:04.751480 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:04.751480 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:04.751480 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:04.751480 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:04.751480 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:04.751480 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:04.751480 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:04.752424 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:04.752424 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:04.752424 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:04.752424 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:04.752424 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:04.752424 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:04.752424 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:04.752424 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:04.752424 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:04.752424 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:04.752424 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:04.752424 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:04.752424 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:04.752424 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:04.752424 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:04.752424 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:04.753472 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:04.753472 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:04.753472 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:04.753472 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:04.753472 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:04.753472 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:04.753472 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:04.753472 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:04.753472 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:04.753472 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:04.753472 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:24:04.753472 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:04.753472 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 159/309] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:24:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:04.758780 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1495f96d-90e5-43ad-8b1f-5e07561bd861 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:04.857881 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce754932-dbde-4c6a-8da9-5f56ca8053ef None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:04.890466 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:04.904589 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd47dba7-b56f-4ef6-a55a-7ad9d64b265f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:04.905381 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd47dba7-b56f-4ef6-a55a-7ad9d64b265f None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:04.905663 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd47dba7-b56f-4ef6-a55a-7ad9d64b265f None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:04.905936 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd47dba7-b56f-4ef6-a55a-7ad9d64b265f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:04.913404 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1495f96d-90e5-43ad-8b1f-5e07561bd861 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:04.917600 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd47dba7-b56f-4ef6-a55a-7ad9d64b265f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:04.925363 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-fd47dba7-b56f-4ef6-a55a-7ad9d64b265f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:24:04.925363 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:04.925363 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:04.925363 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:04.925363 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:04.925363 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:04.925363 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:04.925363 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:04.925363 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:04.925363 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:04.925363 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:04.925363 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:04.925363 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:04.925363 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:04.925363 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:04.925363 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:04.926266 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:04.926266 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:04.926266 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:04.926266 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:04.926266 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:04.926266 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:04.926266 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:04.926266 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:04.926266 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:04.926266 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:04.926266 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:04.926266 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:04.926266 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:04.926266 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:04.926266 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:04.926266 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:04.927053 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:04.927053 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:04.927053 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:04.927053 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:04.927053 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:04.927053 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:04.927053 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:04.927053 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:04.927053 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:04.927053 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:04.927053 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:24:04.927053 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:04.931589 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 151/310] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:24:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:24:04.933151 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd47dba7-b56f-4ef6-a55a-7ad9d64b265f None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:04.939546 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:04.964787 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ac27c8-003c-4b36-b46b-d105bfddc4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:04.966018 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac27c8-003c-4b36-b46b-d105bfddc4be None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:04.966926 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac27c8-003c-4b36-b46b-d105bfddc4be None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:04.967360 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac27c8-003c-4b36-b46b-d105bfddc4be None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:04.968581 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ac27c8-003c-4b36-b46b-d105bfddc4be None admin] RBAC: Adding query filter params (name=member) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:04.969301 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ac27c8-003c-4b36-b46b-d105bfddc4be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:04.971742 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ac27c8-003c-4b36-b46b-d105bfddc4be None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:04.983852 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 160/311] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 18 16:24:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:04.989869 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ac27c8-003c-4b36-b46b-d105bfddc4be None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:05.062141 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd47dba7-b56f-4ef6-a55a-7ad9d64b265f None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:05.101408 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda86299-cbf1-4c2e-874a-9f7d9aaa82e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:05.102086 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda86299-cbf1-4c2e-874a-9f7d9aaa82e8 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:05.102371 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda86299-cbf1-4c2e-874a-9f7d9aaa82e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:05.102896 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda86299-cbf1-4c2e-874a-9f7d9aaa82e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:05.104277 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda86299-cbf1-4c2e-874a-9f7d9aaa82e8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:05.104588 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda86299-cbf1-4c2e-874a-9f7d9aaa82e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:05.106734 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda86299-cbf1-4c2e-874a-9f7d9aaa82e8 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:05.114361 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ac27c8-003c-4b36-b46b-d105bfddc4be None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:05.118228 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 152/312] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 18 16:24:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:05.124251 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda86299-cbf1-4c2e-874a-9f7d9aaa82e8 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:05.156286 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d17ab12-7856-404e-bd05-18565876de5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:05.158160 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d17ab12-7856-404e-bd05-18565876de5e None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:05.158870 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d17ab12-7856-404e-bd05-18565876de5e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:05.159398 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d17ab12-7856-404e-bd05-18565876de5e None admin] PATH_INFO: `/v3/users/02cd09c9ba5341508f7ef7e7e6c96f8a` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:05.161001 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d17ab12-7856-404e-bd05-18565876de5e None admin] RBAC: Authorizing `identity:get_user(user_id=02cd09c9ba5341508f7ef7e7e6c96f8a)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:05.162859 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d17ab12-7856-404e-bd05-18565876de5e None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:05.170823 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 161/313] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 18 16:24:04 2022] GET /identity/v3/users/02cd09c9ba5341508f7ef7e7e6c96f8a => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:05.172129 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d17ab12-7856-404e-bd05-18565876de5e None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:05.188197 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:05.232596 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:05.262716 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda86299-cbf1-4c2e-874a-9f7d9aaa82e8 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:05.291161 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d17ab12-7856-404e-bd05-18565876de5e None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:05.308473 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a553ec3-b7da-4310-b84f-e03657b33870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:05.309368 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a553ec3-b7da-4310-b84f-e03657b33870 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:05.309914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a553ec3-b7da-4310-b84f-e03657b33870 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:05.310201 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a553ec3-b7da-4310-b84f-e03657b33870 None admin] PATH_INFO: `/v3/users/fa41d8cbfdd64a5080b4ced87f44bc86` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:05.311713 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a553ec3-b7da-4310-b84f-e03657b33870 None admin] RBAC: Authorizing `identity:get_user(user_id=fa41d8cbfdd64a5080b4ced87f44bc86)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:05.313589 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a553ec3-b7da-4310-b84f-e03657b33870 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:05.319631 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 153/314] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 18 16:24:05 2022] GET /identity/v3/users/fa41d8cbfdd64a5080b4ced87f44bc86 => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:05.323558 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a553ec3-b7da-4310-b84f-e03657b33870 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:05.339556 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81da8c05-e0a4-4454-8942-6f2f63da62e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:05.340545 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da8c05-e0a4-4454-8942-6f2f63da62e7 None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:05.340833 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da8c05-e0a4-4454-8942-6f2f63da62e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:05.341113 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da8c05-e0a4-4454-8942-6f2f63da62e7 None admin] PATH_INFO: `/v3/system/users/02cd09c9ba5341508f7ef7e7e6c96f8a/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:05.342784 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81da8c05-e0a4-4454-8942-6f2f63da62e7 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=02cd09c9ba5341508f7ef7e7e6c96f8a, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:05.344872 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81da8c05-e0a4-4454-8942-6f2f63da62e7 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:05.432984 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a553ec3-b7da-4310-b84f-e03657b33870 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:05.450801 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 162/315] 10.222.0.26 () {58 vars in 1334 bytes} [Mon Jul 18 16:24:05 2022] PUT /identity/v3/system/users/02cd09c9ba5341508f7ef7e7e6c96f8a/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:24:05.455937 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:05.481264 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c1d836-8175-468a-897e-4d65213bf7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:05.481943 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1d836-8175-468a-897e-4d65213bf7be None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:05.482269 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1d836-8175-468a-897e-4d65213bf7be None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:05.482576 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1d836-8175-468a-897e-4d65213bf7be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:05.567555 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c1d836-8175-468a-897e-4d65213bf7be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:05.569071 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c1d836-8175-468a-897e-4d65213bf7be None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:05.695647 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 154/316] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:24:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:24:05.696588 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81da8c05-e0a4-4454-8942-6f2f63da62e7 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:05.807295 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81da8c05-e0a4-4454-8942-6f2f63da62e7 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:05.821203 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:05.845583 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac9d26-2514-4389-aa0f-45056f250dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:05.846451 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac9d26-2514-4389-aa0f-45056f250dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:05.846737 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac9d26-2514-4389-aa0f-45056f250dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:05.847017 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac9d26-2514-4389-aa0f-45056f250dd9 None admin] PATH_INFO: `/v3/projects/cc6e57482c554178aa9b830aa0b83a96` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:05.848238 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeac9d26-2514-4389-aa0f-45056f250dd9 None admin] RBAC: Authorizing `identity:get_project(project_id=cc6e57482c554178aa9b830aa0b83a96)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:05.849683 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeac9d26-2514-4389-aa0f-45056f250dd9 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:05.851647 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 163/317] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:24:05 2022] GET /identity/v3/projects/cc6e57482c554178aa9b830aa0b83a96 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:05.858502 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c1d836-8175-468a-897e-4d65213bf7be None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:05.878875 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:05.966802 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c1d836-8175-468a-897e-4d65213bf7be None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:06.004023 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a244b-92f8-4802-9f8b-67b0fd080b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:06.004686 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a244b-92f8-4802-9f8b-67b0fd080b65 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:06.004972 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a244b-92f8-4802-9f8b-67b0fd080b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:06.005251 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a244b-92f8-4802-9f8b-67b0fd080b65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:06.006093 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699a244b-92f8-4802-9f8b-67b0fd080b65 None admin] RBAC: Adding query filter params (role.id=2ba8d51abb6e4efa872c82ac8c9ba4a8, scope.project.id=cc6e57482c554178aa9b830aa0b83a96, user.id=fa41d8cbfdd64a5080b4ced87f44bc86) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:06.006404 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699a244b-92f8-4802-9f8b-67b0fd080b65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:06.007855 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699a244b-92f8-4802-9f8b-67b0fd080b65 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:06.014812 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 155/318] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 18 16:24:05 2022] GET /identity/v3/role_assignments?user.id=fa41d8cbfdd64a5080b4ced87f44bc86&scope.project.id=cc6e57482c554178aa9b830aa0b83a96&role.id=2ba8d51abb6e4efa872c82ac8c9ba4a8 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:06.561555 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c14b0ff-36ce-4970-83c2-67508c88060a None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:06.562180 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c14b0ff-36ce-4970-83c2-67508c88060a None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:06.562180 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c14b0ff-36ce-4970-83c2-67508c88060a None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:06.563664 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 164/319] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:06.569481 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e625c9e8-9238-490b-b4b8-24f8bf8da8da None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:06.569952 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e625c9e8-9238-490b-b4b8-24f8bf8da8da None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:06.570052 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e625c9e8-9238-490b-b4b8-24f8bf8da8da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:06.577283 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f1020-bed6-4112-b414-61e59b197e78 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:06.577581 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f1020-bed6-4112-b414-61e59b197e78 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:06.577863 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f1020-bed6-4112-b414-61e59b197e78 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:06.579302 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 165/320] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:06.584724 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170daf7-23ed-46cf-8076-966a4b0d98a0 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:06.585018 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170daf7-23ed-46cf-8076-966a4b0d98a0 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:06.585299 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170daf7-23ed-46cf-8076-966a4b0d98a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:06.587714 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-e625c9e8-9238-490b-b4b8-24f8bf8da8da None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:06.589313 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-e625c9e8-9238-490b-b4b8-24f8bf8da8da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:06.602551 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-d170daf7-23ed-46cf-8076-966a4b0d98a0 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:06.603854 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d170daf7-23ed-46cf-8076-966a4b0d98a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:06.668986 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 156/321] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:06.685667 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 166/322] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:24:06.692802 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e625c9e8-9238-490b-b4b8-24f8bf8da8da None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:06.705031 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:06.707294 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d170daf7-23ed-46cf-8076-966a4b0d98a0 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:06.802228 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e625c9e8-9238-490b-b4b8-24f8bf8da8da None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:06.820836 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d170daf7-23ed-46cf-8076-966a4b0d98a0 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:06.836531 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:06.848804 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19983a6-ba2f-4af2-9b98-f3502700f102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:06.849797 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19983a6-ba2f-4af2-9b98-f3502700f102 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:06.850223 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19983a6-ba2f-4af2-9b98-f3502700f102 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:06.851367 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19983a6-ba2f-4af2-9b98-f3502700f102 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:06.852319 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19983a6-ba2f-4af2-9b98-f3502700f102 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:06.854734 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19983a6-ba2f-4af2-9b98-f3502700f102 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:06.856432 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 157/323] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:06.862019 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19983a6-ba2f-4af2-9b98-f3502700f102 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:06.867733 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a359f1a-ed30-402e-bd7e-40491d64b417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:06.868537 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a359f1a-ed30-402e-bd7e-40491d64b417 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:06.868878 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a359f1a-ed30-402e-bd7e-40491d64b417 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:06.869230 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a359f1a-ed30-402e-bd7e-40491d64b417 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:06.881380 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a359f1a-ed30-402e-bd7e-40491d64b417 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:06.889810 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-9a359f1a-ed30-402e-bd7e-40491d64b417 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:24:06.889810 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:06.889810 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:06.889810 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:06.889810 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:06.889810 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:06.889810 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:06.889810 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:06.889810 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:06.889810 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:06.889810 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:06.889810 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:06.889810 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:06.889810 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:06.889810 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:06.889810 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:06.890782 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:06.890782 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:06.890782 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:06.890782 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:06.890782 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:06.890782 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:06.890782 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:06.890782 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:06.890782 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:06.890782 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:06.890782 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:06.890782 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:06.890782 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:06.890782 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:06.890782 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:06.890782 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:06.896184 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:06.896184 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:06.896184 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:06.896184 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:06.896184 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:06.896184 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:06.896184 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:06.896184 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:06.896184 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:06.896184 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:06.896184 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:24:06.896184 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:06.896184 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 167/324] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:24:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:24:06.896856 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a359f1a-ed30-402e-bd7e-40491d64b417 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:06.952914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:06.976245 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19983a6-ba2f-4af2-9b98-f3502700f102 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:07.006523 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a359f1a-ed30-402e-bd7e-40491d64b417 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:07.017658 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9671ccdb-4ac7-486e-91d6-f758361088eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:07.018372 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9671ccdb-4ac7-486e-91d6-f758361088eb None admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:07.018623 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9671ccdb-4ac7-486e-91d6-f758361088eb None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:07.018903 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9671ccdb-4ac7-486e-91d6-f758361088eb None admin] PATH_INFO: `/v3/groups` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:07.019620 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9671ccdb-4ac7-486e-91d6-f758361088eb None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:07.021328 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9671ccdb-4ac7-486e-91d6-f758361088eb None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:07.040255 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d2339d-1dc6-4ba0-9d75-bedd3f74ee95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:07.041465 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d2339d-1dc6-4ba0-9d75-bedd3f74ee95 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:07.041981 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d2339d-1dc6-4ba0-9d75-bedd3f74ee95 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:07.042339 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d2339d-1dc6-4ba0-9d75-bedd3f74ee95 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:07.043104 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d2339d-1dc6-4ba0-9d75-bedd3f74ee95 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:07.043409 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d2339d-1dc6-4ba0-9d75-bedd3f74ee95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:07.044877 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d2339d-1dc6-4ba0-9d75-bedd3f74ee95 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:07.053883 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 168/325] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 18 16:24:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:07.060176 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d2339d-1dc6-4ba0-9d75-bedd3f74ee95 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:07.090956 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:07.136900 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 158/326] 10.222.0.26 () {60 vars in 1134 bytes} [Mon Jul 18 16:24:06 2022] POST /identity/v3/groups => generated 226 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:24:07.166174 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d2339d-1dc6-4ba0-9d75-bedd3f74ee95 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:07.209996 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9e307a-a0b9-4c18-a3be-a59cc2b06f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:07.210945 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9e307a-a0b9-4c18-a3be-a59cc2b06f0f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:07.211278 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9e307a-a0b9-4c18-a3be-a59cc2b06f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:07.211540 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9e307a-a0b9-4c18-a3be-a59cc2b06f0f None admin] PATH_INFO: `/v3/users/cd8962c375294db8a7b38e35e75bbb52` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:07.212965 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9e307a-a0b9-4c18-a3be-a59cc2b06f0f None admin] RBAC: Authorizing `identity:get_user(user_id=cd8962c375294db8a7b38e35e75bbb52)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:07.214580 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9e307a-a0b9-4c18-a3be-a59cc2b06f0f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:07.216591 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 169/327] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 18 16:24:07 2022] GET /identity/v3/users/cd8962c375294db8a7b38e35e75bbb52 => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:07.224885 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9671ccdb-4ac7-486e-91d6-f758361088eb None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:07.337031 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9671ccdb-4ac7-486e-91d6-f758361088eb None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:07.373610 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d2a9be-d7f9-4c41-ac15-d9569cb9ce3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:07.374438 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d2a9be-d7f9-4c41-ac15-d9569cb9ce3d None admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:07.374722 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d2a9be-d7f9-4c41-ac15-d9569cb9ce3d None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:07.374993 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d2a9be-d7f9-4c41-ac15-d9569cb9ce3d None admin] PATH_INFO: `/v3/system/users/cd8962c375294db8a7b38e35e75bbb52/roles/2ba8d51abb6e4efa872c82ac8c9ba4a8` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:07.376546 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d2a9be-d7f9-4c41-ac15-d9569cb9ce3d None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=cd8962c375294db8a7b38e35e75bbb52, role_id=2ba8d51abb6e4efa872c82ac8c9ba4a8)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:07.377939 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d2a9be-d7f9-4c41-ac15-d9569cb9ce3d None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:07.451104 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 159/328] 10.222.0.26 () {58 vars in 1334 bytes} [Mon Jul 18 16:24:07 2022] PUT /identity/v3/system/users/cd8962c375294db8a7b38e35e75bbb52/roles/2ba8d51abb6e4efa872c82ac8c9ba4a8 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:24:07.617378 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24552e84-bfaf-4ede-b54c-caf2be1f759b None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:07.617913 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24552e84-bfaf-4ede-b54c-caf2be1f759b None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:07.617913 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24552e84-bfaf-4ede-b54c-caf2be1f759b None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:07.619850 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 170/329] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:07.625461 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e80cb-c36a-43c3-bdae-0cec83373c45 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:07.626357 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e80cb-c36a-43c3-bdae-0cec83373c45 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:07.626835 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e80cb-c36a-43c3-bdae-0cec83373c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:07.632424 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:07.646873 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-773e80cb-c36a-43c3-bdae-0cec83373c45 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:07.648111 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-773e80cb-c36a-43c3-bdae-0cec83373c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:07.731366 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 160/330] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:24:07.758576 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24552e84-bfaf-4ede-b54c-caf2be1f759b None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:07.905119 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:07.909441 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24552e84-bfaf-4ede-b54c-caf2be1f759b None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:07.956107 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a725f8e-0d74-4b97-b8ed-a7e7a85fd872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:07.956924 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a725f8e-0d74-4b97-b8ed-a7e7a85fd872 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:07.957205 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a725f8e-0d74-4b97-b8ed-a7e7a85fd872 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:07.957489 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a725f8e-0d74-4b97-b8ed-a7e7a85fd872 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:07.972839 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a725f8e-0d74-4b97-b8ed-a7e7a85fd872 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:07.981667 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-4a725f8e-0d74-4b97-b8ed-a7e7a85fd872 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:24:07.981667 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:07.981667 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:07.981667 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:07.981667 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:07.981667 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:07.981667 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:07.981667 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:07.981667 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:07.981667 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:07.981667 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:07.981667 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:07.981667 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:07.981667 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:07.981667 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:07.981667 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:07.982613 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:07.982613 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:07.982613 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:07.982613 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:07.982613 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:07.982613 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:07.982613 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:07.982613 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:07.982613 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:07.982613 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:07.982613 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:07.982613 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:07.982613 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:07.982613 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:07.982613 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:07.982613 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:07.983600 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:07.983600 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:07.983600 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:07.983600 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:07.983600 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:07.983600 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:07.983600 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:07.983600 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:07.983600 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:07.983600 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:07.983600 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:24:07.983600 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:07.983600 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 171/331] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:24:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:07.991193 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773e80cb-c36a-43c3-bdae-0cec83373c45 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:08.113765 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773e80cb-c36a-43c3-bdae-0cec83373c45 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:08.147929 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:08.153642 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9a5c0f-5526-4c3c-b801-1e08ead66403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:08.154416 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a5c0f-5526-4c3c-b801-1e08ead66403 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:08.154709 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a5c0f-5526-4c3c-b801-1e08ead66403 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:08.154987 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a5c0f-5526-4c3c-b801-1e08ead66403 None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:08.165642 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9a5c0f-5526-4c3c-b801-1e08ead66403 None admin] RBAC: Adding query filter params (name=member) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:08.165642 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9a5c0f-5526-4c3c-b801-1e08ead66403 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:08.165642 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9a5c0f-5526-4c3c-b801-1e08ead66403 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:08.167015 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 161/332] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 18 16:24:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:08.172545 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a725f8e-0d74-4b97-b8ed-a7e7a85fd872 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:08.283381 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a725f8e-0d74-4b97-b8ed-a7e7a85fd872 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:08.320224 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cfefee-f1bc-4c84-a67f-b0b6bb036dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:08.321106 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cfefee-f1bc-4c84-a67f-b0b6bb036dab None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:08.321443 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cfefee-f1bc-4c84-a67f-b0b6bb036dab None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:08.321730 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cfefee-f1bc-4c84-a67f-b0b6bb036dab None admin] PATH_INFO: `/v3/users/02cd09c9ba5341508f7ef7e7e6c96f8a` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:08.323642 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cfefee-f1bc-4c84-a67f-b0b6bb036dab None admin] RBAC: Authorizing `identity:get_user(user_id=02cd09c9ba5341508f7ef7e7e6c96f8a)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:08.325094 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cfefee-f1bc-4c84-a67f-b0b6bb036dab None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:08.327162 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 172/333] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 18 16:24:08 2022] GET /identity/v3/users/02cd09c9ba5341508f7ef7e7e6c96f8a => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:08.332813 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9a5c0f-5526-4c3c-b801-1e08ead66403 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:08.458070 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9a5c0f-5526-4c3c-b801-1e08ead66403 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:08.496899 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3484fad0-13cd-4f24-8f4f-10167aebdb1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:08.497570 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3484fad0-13cd-4f24-8f4f-10167aebdb1d None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:08.497853 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3484fad0-13cd-4f24-8f4f-10167aebdb1d None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:08.498132 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3484fad0-13cd-4f24-8f4f-10167aebdb1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:08.498973 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3484fad0-13cd-4f24-8f4f-10167aebdb1d None admin] RBAC: Adding query filter params (role.id=aa0552e8ba924f5d9cb1740da7d4a4dc, user.id=02cd09c9ba5341508f7ef7e7e6c96f8a, scope.system=all) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:08.499287 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3484fad0-13cd-4f24-8f4f-10167aebdb1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:08.500785 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3484fad0-13cd-4f24-8f4f-10167aebdb1d None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:08.518395 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 162/334] 10.222.0.26 () {56 vars in 1308 bytes} [Mon Jul 18 16:24:08 2022] GET /identity/v3/role_assignments?user.id=02cd09c9ba5341508f7ef7e7e6c96f8a&scope.system=all&role.id=aa0552e8ba924f5d9cb1740da7d4a4dc => generated 510 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:09.778929 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58e81a6-cc72-4c95-8e74-d8a88c2d6d3e None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:09.779582 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58e81a6-cc72-4c95-8e74-d8a88c2d6d3e None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:09.779582 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58e81a6-cc72-4c95-8e74-d8a88c2d6d3e None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:09.781211 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 173/335] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:09.786308 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941b3cb-53b1-472a-9188-4490d9f43e64 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:09.787170 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941b3cb-53b1-472a-9188-4490d9f43e64 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:09.787679 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941b3cb-53b1-472a-9188-4490d9f43e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:09.808176 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-3941b3cb-53b1-472a-9188-4490d9f43e64 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:09.809744 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-3941b3cb-53b1-472a-9188-4490d9f43e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:09.880701 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:09.903057 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 163/336] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:09.906887 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc6b6b-4bcd-4856-8312-d00a71b3920d None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:09.907181 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc6b6b-4bcd-4856-8312-d00a71b3920d None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:09.907465 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc6b6b-4bcd-4856-8312-d00a71b3920d None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:09.909005 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 174/337] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:09.914290 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6764ac2-a7ea-492a-8f15-0f277f495a1c None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:09.914588 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6764ac2-a7ea-492a-8f15-0f277f495a1c None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:09.914872 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6764ac2-a7ea-492a-8f15-0f277f495a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:09.937384 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-e6764ac2-a7ea-492a-8f15-0f277f495a1c None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:09.938579 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-e6764ac2-a7ea-492a-8f15-0f277f495a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:09.950482 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bc6b6b-4bcd-4856-8312-d00a71b3920d None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:10.016187 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:10.050072 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 164/338] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:10.095610 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6764ac2-a7ea-492a-8f15-0f277f495a1c None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:10.201075 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bc6b6b-4bcd-4856-8312-d00a71b3920d None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:10.275617 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb6713b-7d46-4796-ab44-c226310e4f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:10.277428 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6713b-7d46-4796-ab44-c226310e4f59 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:10.278099 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6713b-7d46-4796-ab44-c226310e4f59 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:10.278755 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6713b-7d46-4796-ab44-c226310e4f59 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:10.306227 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb6713b-7d46-4796-ab44-c226310e4f59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:10.315107 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-6fb6713b-7d46-4796-ab44-c226310e4f59 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:24:10.315107 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:10.315107 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:10.315107 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:10.315107 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:10.315107 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:10.315107 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:10.315107 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:10.315107 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:10.315107 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:10.315107 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:10.315107 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:10.315107 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:10.315107 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:10.315107 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:10.315107 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:10.316286 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:10.316286 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:10.316286 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:10.316286 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:10.316286 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:10.316286 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:10.316286 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:10.316286 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:10.316286 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:10.316286 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:10.316286 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:10.316286 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:10.316286 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:10.316286 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:10.316286 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:10.316286 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:10.317114 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:10.317114 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:10.317114 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:10.317114 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:10.317114 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:10.317114 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:10.317114 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:10.317114 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:10.317114 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:10.317114 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:10.317114 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:24:10.317114 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:10.323941 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 175/339] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:24:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:10.330140 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb6713b-7d46-4796-ab44-c226310e4f59 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:10.345813 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6764ac2-a7ea-492a-8f15-0f277f495a1c None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:10.368539 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:10.427374 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3476af00-62b0-497b-a7f6-5a164a8b5af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:10.428390 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3476af00-62b0-497b-a7f6-5a164a8b5af3 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:10.428792 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3476af00-62b0-497b-a7f6-5a164a8b5af3 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:10.429168 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3476af00-62b0-497b-a7f6-5a164a8b5af3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:10.460529 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3476af00-62b0-497b-a7f6-5a164a8b5af3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:10.473564 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-3476af00-62b0-497b-a7f6-5a164a8b5af3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:24:10.473564 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:10.473564 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:10.473564 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:10.473564 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:10.473564 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:10.473564 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:10.473564 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:10.473564 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:10.473564 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:10.473564 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:10.473564 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:10.473564 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:10.473564 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:10.473564 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:10.473564 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:10.474489 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:10.474489 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:10.474489 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:10.474489 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:10.474489 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:10.474489 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:10.474489 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:10.474489 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:10.474489 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:10.474489 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:10.474489 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:10.474489 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:10.474489 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:10.474489 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:10.474489 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:10.474489 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:10.475281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:10.475281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:10.475281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:10.475281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:10.475281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:10.475281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:10.475281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:10.475281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:10.475281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:10.475281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:10.475281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:24:10.475281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:10.475281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 165/340] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 18 16:24:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:10.482193 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3476af00-62b0-497b-a7f6-5a164a8b5af3 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:10.533818 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:10.555352 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb6713b-7d46-4796-ab44-c226310e4f59 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:10.621502 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72141ff9-f8c1-4589-a5b0-38788989735f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:10.622103 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72141ff9-f8c1-4589-a5b0-38788989735f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:10.622390 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72141ff9-f8c1-4589-a5b0-38788989735f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:10.622682 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72141ff9-f8c1-4589-a5b0-38788989735f None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:10.623426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72141ff9-f8c1-4589-a5b0-38788989735f None admin] RBAC: Adding query filter params (name=member) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:10.623755 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72141ff9-f8c1-4589-a5b0-38788989735f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:10.625166 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72141ff9-f8c1-4589-a5b0-38788989735f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:10.645823 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 176/341] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 18 16:24:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:10.653410 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72141ff9-f8c1-4589-a5b0-38788989735f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:10.724264 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3476af00-62b0-497b-a7f6-5a164a8b5af3 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:10.793282 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d939ba7c-ff45-4091-88e3-accc467cf16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:10.794098 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939ba7c-ff45-4091-88e3-accc467cf16a None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:10.794265 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939ba7c-ff45-4091-88e3-accc467cf16a None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:10.794550 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939ba7c-ff45-4091-88e3-accc467cf16a None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:10.795393 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d939ba7c-ff45-4091-88e3-accc467cf16a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:10.795733 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d939ba7c-ff45-4091-88e3-accc467cf16a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:10.797620 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d939ba7c-ff45-4091-88e3-accc467cf16a None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:10.813737 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 166/342] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:24:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:10.815205 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb5067-5d8f-4073-a02b-19ee2099d61b None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:10.815660 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb5067-5d8f-4073-a02b-19ee2099d61b None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:10.815948 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb5067-5d8f-4073-a02b-19ee2099d61b None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:10.817450 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 167/343] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:10.821516 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bb5067-5d8f-4073-a02b-19ee2099d61b None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:10.865312 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:10.913784 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72141ff9-f8c1-4589-a5b0-38788989735f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:10.943689 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bb5067-5d8f-4073-a02b-19ee2099d61b None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:10.981274 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec74842a-b82a-4b0e-92e7-fe4f0d4c0022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:10.981927 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec74842a-b82a-4b0e-92e7-fe4f0d4c0022 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:10.982212 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec74842a-b82a-4b0e-92e7-fe4f0d4c0022 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:10.982491 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec74842a-b82a-4b0e-92e7-fe4f0d4c0022 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:11.000245 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0223bf-2065-4379-972c-5079e995d23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:11.001294 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0223bf-2065-4379-972c-5079e995d23c None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:11.001896 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0223bf-2065-4379-972c-5079e995d23c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:11.002392 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0223bf-2065-4379-972c-5079e995d23c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:11.057461 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:11.064188 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec74842a-b82a-4b0e-92e7-fe4f0d4c0022 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:11.065809 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec74842a-b82a-4b0e-92e7-fe4f0d4c0022 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:11.107072 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:11.141108 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0223bf-2065-4379-972c-5079e995d23c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:11.143581 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0223bf-2065-4379-972c-5079e995d23c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:11.188475 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 168/344] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:24:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:24:11.190248 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea32d1fe-d8f2-4e6e-a409-57108a3fa606 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:11.190813 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea32d1fe-d8f2-4e6e-a409-57108a3fa606 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:11.191345 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea32d1fe-d8f2-4e6e-a409-57108a3fa606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:11.209759 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-ea32d1fe-d8f2-4e6e-a409-57108a3fa606 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:11.211288 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-ea32d1fe-d8f2-4e6e-a409-57108a3fa606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:11.277521 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 177/345] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:24:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 624 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 16:24:11.278769 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6a3eb-9302-4abd-9529-4e6507f4011e None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:11.279059 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6a3eb-9302-4abd-9529-4e6507f4011e None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:11.279339 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6a3eb-9302-4abd-9529-4e6507f4011e None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:11.282898 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 178/346] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:11.282898 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f6a3eb-9302-4abd-9529-4e6507f4011e None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:11.332469 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 169/347] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:11.334091 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba641da-9ddb-4487-84e5-cd1cddd61572 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:11.334578 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba641da-9ddb-4487-84e5-cd1cddd61572 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:11.335128 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba641da-9ddb-4487-84e5-cd1cddd61572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:11.352647 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-bba641da-9ddb-4487-84e5-cd1cddd61572 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:11.355080 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-bba641da-9ddb-4487-84e5-cd1cddd61572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:11.361105 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:11.412570 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f6a3eb-9302-4abd-9529-4e6507f4011e None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:11.431186 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:11.442980 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 170/348] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:11.442980 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba641da-9ddb-4487-84e5-cd1cddd61572 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:11.459260 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1274cac7-42a7-47cb-b9c2-a271b5dc5ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:11.460566 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1274cac7-42a7-47cb-b9c2-a271b5dc5ace None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:11.460855 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1274cac7-42a7-47cb-b9c2-a271b5dc5ace None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:11.461131 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1274cac7-42a7-47cb-b9c2-a271b5dc5ace None admin] PATH_INFO: `/v3/projects/04afdb2deaea49078a14a588c9d6de7e` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:11.462814 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1274cac7-42a7-47cb-b9c2-a271b5dc5ace None admin] RBAC: Authorizing `identity:get_project(project_id=04afdb2deaea49078a14a588c9d6de7e)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:11.465704 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1274cac7-42a7-47cb-b9c2-a271b5dc5ace None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:11.467881 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 179/349] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:24:11 2022] GET /identity/v3/projects/04afdb2deaea49078a14a588c9d6de7e => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:11.468696 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1274cac7-42a7-47cb-b9c2-a271b5dc5ace None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:11.570183 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba641da-9ddb-4487-84e5-cd1cddd61572 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:11.613647 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f4ff07-8a7e-4145-882f-ff329109814d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:11.614904 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f4ff07-8a7e-4145-882f-ff329109814d None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:11.615399 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f4ff07-8a7e-4145-882f-ff329109814d None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:11.616066 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f4ff07-8a7e-4145-882f-ff329109814d None admin] PATH_INFO: `/v3/projects/04afdb2deaea49078a14a588c9d6de7e` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:11.617611 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f4ff07-8a7e-4145-882f-ff329109814d None admin] RBAC: Authorizing `identity:get_project(project_id=04afdb2deaea49078a14a588c9d6de7e)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:11.619971 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f4ff07-8a7e-4145-882f-ff329109814d None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:11.623894 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 171/350] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:24:11 2022] GET /identity/v3/projects/04afdb2deaea49078a14a588c9d6de7e => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:11.624164 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1274cac7-42a7-47cb-b9c2-a271b5dc5ace None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:11.625179 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f4ff07-8a7e-4145-882f-ff329109814d None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:11.663891 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:11.677022 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bd92db-e359-4bbd-8bd4-1fba482a015f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:11.678107 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd92db-e359-4bbd-8bd4-1fba482a015f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:11.678391 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd92db-e359-4bbd-8bd4-1fba482a015f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:11.678666 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd92db-e359-4bbd-8bd4-1fba482a015f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:11.696163 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bd92db-e359-4bbd-8bd4-1fba482a015f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:11.707784 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-f8bd92db-e359-4bbd-8bd4-1fba482a015f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:24:11.707784 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:11.707784 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:11.707784 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:11.707784 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:11.707784 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:11.707784 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:11.707784 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:11.707784 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:11.707784 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:11.707784 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:11.707784 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:11.707784 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:11.707784 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:11.707784 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:11.707784 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:11.708786 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:11.708786 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:11.708786 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:11.708786 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:11.708786 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:11.708786 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:11.708786 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:11.708786 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:11.708786 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:11.708786 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:11.708786 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:11.708786 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:11.708786 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:11.708786 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:11.708786 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:11.708786 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:11.709614 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:11.709614 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:11.709614 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:11.709614 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:11.709614 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:11.709614 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:11.709614 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:11.709614 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:11.709614 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:11.709614 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:11.709614 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:24:11.709614 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:11.709614 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 180/351] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:24:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:11.711074 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bd92db-e359-4bbd-8bd4-1fba482a015f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:11.721109 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:11.780499 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f4ff07-8a7e-4145-882f-ff329109814d None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:11.831993 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2267030b-ed1e-4ad0-b9c6-c985dd3e46a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:11.833219 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2267030b-ed1e-4ad0-b9c6-c985dd3e46a9 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:11.833618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2267030b-ed1e-4ad0-b9c6-c985dd3e46a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:11.833922 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2267030b-ed1e-4ad0-b9c6-c985dd3e46a9 None admin] PATH_INFO: `/v3/groups/66ae82ee5b514025819f0b50ed97d9c9` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:11.847229 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2267030b-ed1e-4ad0-b9c6-c985dd3e46a9 None admin] RBAC: Authorizing `identity:get_group(group_id=66ae82ee5b514025819f0b50ed97d9c9)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:11.848765 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2267030b-ed1e-4ad0-b9c6-c985dd3e46a9 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:11.850722 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 172/352] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jul 18 16:24:11 2022] GET /identity/v3/groups/66ae82ee5b514025819f0b50ed97d9c9 => generated 226 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:11.851265 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2267030b-ed1e-4ad0-b9c6-c985dd3e46a9 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:11.858487 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bd92db-e359-4bbd-8bd4-1fba482a015f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:11.898208 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645b1475-c730-4a3a-a88f-c15abf6e72d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:11.899235 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b1475-c730-4a3a-a88f-c15abf6e72d5 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:11.899560 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b1475-c730-4a3a-a88f-c15abf6e72d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:11.899845 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b1475-c730-4a3a-a88f-c15abf6e72d5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:11.917472 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645b1475-c730-4a3a-a88f-c15abf6e72d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:11.927701 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-645b1475-c730-4a3a-a88f-c15abf6e72d5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:24:11.927701 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:11.927701 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:11.927701 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:11.927701 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:11.927701 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:11.927701 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:11.927701 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:11.927701 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:11.927701 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:11.927701 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:11.927701 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:11.927701 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:11.927701 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:11.927701 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:11.927701 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:11.928644 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:11.928644 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:11.928644 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:11.928644 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:11.928644 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:11.928644 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:11.928644 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:11.928644 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:11.928644 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:11.928644 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:11.928644 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:11.928644 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:11.928644 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:11.928644 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:11.928644 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:11.928644 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:11.929433 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:11.929433 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:11.929433 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:11.929433 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:11.929433 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:11.929433 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:11.929433 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:11.929433 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:11.929433 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:11.929433 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:11.929433 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:24:11.929433 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:11.930648 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 181/353] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:24:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:11.930648 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645b1475-c730-4a3a-a88f-c15abf6e72d5 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:11.985177 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:11.987787 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2267030b-ed1e-4ad0-b9c6-c985dd3e46a9 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:12.026799 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:12.036627 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d866fd89-7d3d-4ce4-a3d0-47a29a89d266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:12.037518 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866fd89-7d3d-4ce4-a3d0-47a29a89d266 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:12.037810 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866fd89-7d3d-4ce4-a3d0-47a29a89d266 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:12.044603 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866fd89-7d3d-4ce4-a3d0-47a29a89d266 None admin] PATH_INFO: `/v3/groups/66ae82ee5b514025819f0b50ed97d9c9` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:12.044603 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d866fd89-7d3d-4ce4-a3d0-47a29a89d266 None admin] RBAC: Authorizing `identity:get_group(group_id=66ae82ee5b514025819f0b50ed97d9c9)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:12.044603 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d866fd89-7d3d-4ce4-a3d0-47a29a89d266 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:12.044603 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 173/354] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jul 18 16:24:11 2022] GET /identity/v3/groups/66ae82ee5b514025819f0b50ed97d9c9 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:12.045161 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d866fd89-7d3d-4ce4-a3d0-47a29a89d266 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:12.080400 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645b1475-c730-4a3a-a88f-c15abf6e72d5 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:12.124211 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298f75fd-d407-4b68-9691-311f3b45438e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:12.124893 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f75fd-d407-4b68-9691-311f3b45438e None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:12.125181 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f75fd-d407-4b68-9691-311f3b45438e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:12.125459 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f75fd-d407-4b68-9691-311f3b45438e None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:12.126212 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298f75fd-d407-4b68-9691-311f3b45438e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:12.126527 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298f75fd-d407-4b68-9691-311f3b45438e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:12.127955 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298f75fd-d407-4b68-9691-311f3b45438e None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:12.138658 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 182/355] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 18 16:24:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:12.140062 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298f75fd-d407-4b68-9691-311f3b45438e None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:12.179679 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d866fd89-7d3d-4ce4-a3d0-47a29a89d266 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:12.223143 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbd4942-e2e0-4ba7-997b-9549f5b227fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:12.223855 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd4942-e2e0-4ba7-997b-9549f5b227fc None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:12.224132 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd4942-e2e0-4ba7-997b-9549f5b227fc None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:12.224418 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd4942-e2e0-4ba7-997b-9549f5b227fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:12.225279 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbd4942-e2e0-4ba7-997b-9549f5b227fc None admin] RBAC: Adding query filter params (group.id=66ae82ee5b514025819f0b50ed97d9c9, role.id=729cd9c0f42d47b6a8eee01e5f5a2d28, scope.project.id=04afdb2deaea49078a14a588c9d6de7e) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:12.225593 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbd4942-e2e0-4ba7-997b-9549f5b227fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:12.227059 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbd4942-e2e0-4ba7-997b-9549f5b227fc None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:12.234155 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 174/356] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Jul 18 16:24:12 2022] GET /identity/v3/role_assignments?group.id=66ae82ee5b514025819f0b50ed97d9c9&scope.project.id=04afdb2deaea49078a14a588c9d6de7e&role.id=729cd9c0f42d47b6a8eee01e5f5a2d28 => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:12.235336 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90a61c-6dc0-4fef-b4fe-6cca8543c600 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:12.235593 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90a61c-6dc0-4fef-b4fe-6cca8543c600 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:12.235867 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90a61c-6dc0-4fef-b4fe-6cca8543c600 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:12.237250 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 175/357] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:12.238224 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99755c2-7d43-459a-b130-62940d60d285 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:12.238503 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99755c2-7d43-459a-b130-62940d60d285 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:12.238775 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99755c2-7d43-459a-b130-62940d60d285 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:12.240102 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 176/358] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:12.240831 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99755c2-7d43-459a-b130-62940d60d285 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:12.270192 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:12.273520 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298f75fd-d407-4b68-9691-311f3b45438e None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:12.324857 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579a36ce-f79a-4ff2-91d1-d8e697309bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:12.324857 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a36ce-f79a-4ff2-91d1-d8e697309bef None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:12.324857 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a36ce-f79a-4ff2-91d1-d8e697309bef None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:12.325247 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a36ce-f79a-4ff2-91d1-d8e697309bef None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:12.325778 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579a36ce-f79a-4ff2-91d1-d8e697309bef None admin] RBAC: Adding query filter params (name=member) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:12.326181 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579a36ce-f79a-4ff2-91d1-d8e697309bef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:12.326704 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:12.327538 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579a36ce-f79a-4ff2-91d1-d8e697309bef None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:12.344102 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 183/359] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 18 16:24:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:12.345042 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579a36ce-f79a-4ff2-91d1-d8e697309bef None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:12.414000 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99755c2-7d43-459a-b130-62940d60d285 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:12.459776 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a622ba-e9a1-48cb-85b3-21a520ab49f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:12.460446 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a622ba-e9a1-48cb-85b3-21a520ab49f2 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:12.460730 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a622ba-e9a1-48cb-85b3-21a520ab49f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:12.461013 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a622ba-e9a1-48cb-85b3-21a520ab49f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:12.461860 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a622ba-e9a1-48cb-85b3-21a520ab49f2 None admin] RBAC: Adding query filter params (group.id=66ae82ee5b514025819f0b50ed97d9c9, role.id=aa0552e8ba924f5d9cb1740da7d4a4dc, scope.project.id=04afdb2deaea49078a14a588c9d6de7e) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:12.462167 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a622ba-e9a1-48cb-85b3-21a520ab49f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:12.464287 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a622ba-e9a1-48cb-85b3-21a520ab49f2 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:12.471707 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 177/360] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Jul 18 16:24:12 2022] GET /identity/v3/role_assignments?group.id=66ae82ee5b514025819f0b50ed97d9c9&scope.project.id=04afdb2deaea49078a14a588c9d6de7e&role.id=aa0552e8ba924f5d9cb1740da7d4a4dc => generated 260 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:12.471889 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cbbab5-4614-49d8-852a-0add2fa0feac None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:12.472107 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cbbab5-4614-49d8-852a-0add2fa0feac None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:12.472353 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cbbab5-4614-49d8-852a-0add2fa0feac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:12.495125 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-e3cbbab5-4614-49d8-852a-0add2fa0feac None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:12.496432 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-e3cbbab5-4614-49d8-852a-0add2fa0feac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:12.496678 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579a36ce-f79a-4ff2-91d1-d8e697309bef None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:12.543941 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b5996-f67d-4fd6-a8e6-af2e0f82cc31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:12.544961 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b5996-f67d-4fd6-a8e6-af2e0f82cc31 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:12.545339 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b5996-f67d-4fd6-a8e6-af2e0f82cc31 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:12.545627 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b5996-f67d-4fd6-a8e6-af2e0f82cc31 None admin] PATH_INFO: `/v3/users/cd8962c375294db8a7b38e35e75bbb52` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:12.547322 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b5996-f67d-4fd6-a8e6-af2e0f82cc31 None admin] RBAC: Authorizing `identity:get_user(user_id=cd8962c375294db8a7b38e35e75bbb52)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:12.549047 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b5996-f67d-4fd6-a8e6-af2e0f82cc31 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:12.555604 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 184/361] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 18 16:24:12 2022] GET /identity/v3/users/cd8962c375294db8a7b38e35e75bbb52 => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:12.558060 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5088356-fbac-462d-9dab-aaca55702ab9 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:12.558350 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5088356-fbac-462d-9dab-aaca55702ab9 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:12.558628 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5088356-fbac-462d-9dab-aaca55702ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:12.584986 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-b5088356-fbac-462d-9dab-aaca55702ab9 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:12.586222 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-b5088356-fbac-462d-9dab-aaca55702ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:12.601133 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 178/362] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:12.602519 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cbbab5-4614-49d8-852a-0add2fa0feac None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:12.624765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:12.673309 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:12.715233 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 185/363] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:12.716172 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5088356-fbac-462d-9dab-aaca55702ab9 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:12.760921 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cbbab5-4614-49d8-852a-0add2fa0feac None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:12.806299 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464c6f0e-4da9-4e6d-a245-62acf8e51bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:12.807586 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c6f0e-4da9-4e6d-a245-62acf8e51bb5 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:12.807746 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c6f0e-4da9-4e6d-a245-62acf8e51bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:12.808145 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c6f0e-4da9-4e6d-a245-62acf8e51bb5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:12.862057 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5088356-fbac-462d-9dab-aaca55702ab9 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:12.908934 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f06f50d-589c-4319-9279-1646edfedce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:12.909702 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f06f50d-589c-4319-9279-1646edfedce1 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:12.909989 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f06f50d-589c-4319-9279-1646edfedce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:12.910264 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f06f50d-589c-4319-9279-1646edfedce1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:12.911252 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f06f50d-589c-4319-9279-1646edfedce1 None admin] RBAC: Adding query filter params (role.id=2ba8d51abb6e4efa872c82ac8c9ba4a8, user.id=cd8962c375294db8a7b38e35e75bbb52, scope.system=all) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:12.912466 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f06f50d-589c-4319-9279-1646edfedce1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:12.913333 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f06f50d-589c-4319-9279-1646edfedce1 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:12.915988 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464c6f0e-4da9-4e6d-a245-62acf8e51bb5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:12.918918 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464c6f0e-4da9-4e6d-a245-62acf8e51bb5 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:12.931548 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 186/364] 10.222.0.26 () {56 vars in 1308 bytes} [Mon Jul 18 16:24:12 2022] GET /identity/v3/role_assignments?user.id=cd8962c375294db8a7b38e35e75bbb52&scope.system=all&role.id=2ba8d51abb6e4efa872c82ac8c9ba4a8 => generated 510 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:12.931548 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f06f50d-589c-4319-9279-1646edfedce1 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:12.977608 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:13.021527 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:13.125701 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 179/365] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:24:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:24:13.126923 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464c6f0e-4da9-4e6d-a245-62acf8e51bb5 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:13.157865 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f06f50d-589c-4319-9279-1646edfedce1 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:13.212475 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa5ed19-ca27-4bae-b12b-d02abf24f3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:13.213333 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5ed19-ca27-4bae-b12b-d02abf24f3f4 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:13.213624 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5ed19-ca27-4bae-b12b-d02abf24f3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:13.213901 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5ed19-ca27-4bae-b12b-d02abf24f3f4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:13.231473 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa5ed19-ca27-4bae-b12b-d02abf24f3f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:13.242736 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-efa5ed19-ca27-4bae-b12b-d02abf24f3f4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:24:13.242736 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:13.242736 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:13.242736 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:13.242736 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:13.242736 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:13.242736 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:13.242736 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:13.242736 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:13.242736 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:13.242736 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:13.242736 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:13.242736 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:13.242736 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:13.242736 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:13.242736 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:13.243696 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:13.243696 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:13.243696 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:13.243696 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:13.243696 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:13.243696 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:13.243696 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:13.243696 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:13.243696 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:13.243696 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:13.243696 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:13.243696 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:13.243696 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:13.243696 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:13.243696 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:13.243696 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:13.245108 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:13.245108 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:13.245108 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:13.245108 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:13.245108 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:13.245108 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:13.245108 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:13.245108 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:13.245108 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:13.245108 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:13.245108 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:24:13.245108 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:13.245108 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 187/366] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 18 16:24:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:13.246251 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa5ed19-ca27-4bae-b12b-d02abf24f3f4 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:13.267342 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464c6f0e-4da9-4e6d-a245-62acf8e51bb5 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:13.344669 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce62cbe-ef23-48ee-9c35-6b983e8c5925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:13.346150 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce62cbe-ef23-48ee-9c35-6b983e8c5925 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:13.346980 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce62cbe-ef23-48ee-9c35-6b983e8c5925 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:13.347464 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce62cbe-ef23-48ee-9c35-6b983e8c5925 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:13.368767 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce62cbe-ef23-48ee-9c35-6b983e8c5925 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:13.378091 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:13.378281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-9ce62cbe-ef23-48ee-9c35-6b983e8c5925 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:24:13.378281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:13.378281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:13.378281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:13.378281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:13.378281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:13.378281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:13.378281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:13.378281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:13.378281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:13.378281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:13.378281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:13.378281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:13.378281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:13.378281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:13.378281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:13.379105 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:13.379105 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:13.379105 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:13.379105 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:13.379105 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:13.379105 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:13.379105 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:13.379105 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:13.379105 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:13.379105 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:13.379105 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:13.379105 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:13.379105 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:13.379105 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:13.379105 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:13.379105 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:13.384354 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:13.384354 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:13.384354 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:13.384354 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:13.384354 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:13.384354 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:13.384354 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:13.384354 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:13.384354 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:13.384354 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:13.384354 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:24:13.384354 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:13.384354 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 180/367] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 18 16:24:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:13.384354 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce62cbe-ef23-48ee-9c35-6b983e8c5925 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:13.406364 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:13.412243 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa5ed19-ca27-4bae-b12b-d02abf24f3f4 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:13.451809 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31363806-f65a-4d24-b480-200e2a5bf14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:13.453041 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31363806-f65a-4d24-b480-200e2a5bf14f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:13.453538 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31363806-f65a-4d24-b480-200e2a5bf14f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:13.454014 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31363806-f65a-4d24-b480-200e2a5bf14f None admin] PATH_INFO: `/v3/projects/cc6e57482c554178aa9b830aa0b83a96` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:13.455481 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31363806-f65a-4d24-b480-200e2a5bf14f None admin] RBAC: Authorizing `identity:get_project(project_id=cc6e57482c554178aa9b830aa0b83a96)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:13.457135 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31363806-f65a-4d24-b480-200e2a5bf14f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:13.459049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 188/368] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:24:13 2022] GET /identity/v3/projects/cc6e57482c554178aa9b830aa0b83a96 => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:13.460063 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31363806-f65a-4d24-b480-200e2a5bf14f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:13.507180 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce62cbe-ef23-48ee-9c35-6b983e8c5925 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:13.546314 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f0bf6a-b2ec-464f-8a4c-8524d2ae911b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:13.547079 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0bf6a-b2ec-464f-8a4c-8524d2ae911b None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:13.547370 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0bf6a-b2ec-464f-8a4c-8524d2ae911b None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:13.547707 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0bf6a-b2ec-464f-8a4c-8524d2ae911b None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:13.548537 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f0bf6a-b2ec-464f-8a4c-8524d2ae911b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:13.548848 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f0bf6a-b2ec-464f-8a4c-8524d2ae911b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:13.550309 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f0bf6a-b2ec-464f-8a4c-8524d2ae911b None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:13.561209 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 181/369] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:24:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:13.562028 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f0bf6a-b2ec-464f-8a4c-8524d2ae911b None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:13.581897 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31363806-f65a-4d24-b480-200e2a5bf14f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:13.627951 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1abc3c8-62a6-4c3d-aa8a-27aa8c76411f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:13.628562 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1abc3c8-62a6-4c3d-aa8a-27aa8c76411f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:13.628854 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1abc3c8-62a6-4c3d-aa8a-27aa8c76411f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:13.629138 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1abc3c8-62a6-4c3d-aa8a-27aa8c76411f None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:13.629903 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1abc3c8-62a6-4c3d-aa8a-27aa8c76411f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:13.630287 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1abc3c8-62a6-4c3d-aa8a-27aa8c76411f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:13.632550 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1abc3c8-62a6-4c3d-aa8a-27aa8c76411f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:13.646084 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 189/370] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:24:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:13.647040 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1abc3c8-62a6-4c3d-aa8a-27aa8c76411f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:13.664283 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:13.670060 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:13.694078 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f0bf6a-b2ec-464f-8a4c-8524d2ae911b None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:13.738702 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95939c3-a7d4-4580-aeb4-632f89efc670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:13.740065 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95939c3-a7d4-4580-aeb4-632f89efc670 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:13.740356 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95939c3-a7d4-4580-aeb4-632f89efc670 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:13.740640 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95939c3-a7d4-4580-aeb4-632f89efc670 None admin] PATH_INFO: `/v3/groups/66ae82ee5b514025819f0b50ed97d9c9` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:13.741891 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95939c3-a7d4-4580-aeb4-632f89efc670 None admin] RBAC: Authorizing `identity:get_group(group_id=66ae82ee5b514025819f0b50ed97d9c9)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:13.743361 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95939c3-a7d4-4580-aeb4-632f89efc670 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:13.745919 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 182/371] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jul 18 16:24:13 2022] GET /identity/v3/groups/66ae82ee5b514025819f0b50ed97d9c9 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:13.747156 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95939c3-a7d4-4580-aeb4-632f89efc670 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:13.787317 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1abc3c8-62a6-4c3d-aa8a-27aa8c76411f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:13.859399 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc9e644-8d8c-46b0-81d0-2954e8fbf611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:13.860107 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc9e644-8d8c-46b0-81d0-2954e8fbf611 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:13.860549 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc9e644-8d8c-46b0-81d0-2954e8fbf611 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:13.860972 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc9e644-8d8c-46b0-81d0-2954e8fbf611 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:13.918544 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95939c3-a7d4-4580-aeb4-632f89efc670 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:13.945519 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc9e644-8d8c-46b0-81d0-2954e8fbf611 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:13.946993 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc9e644-8d8c-46b0-81d0-2954e8fbf611 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:13.960199 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:13.990925 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c481f558-46ba-4445-9e4b-c0ab9ed393a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:13.992028 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481f558-46ba-4445-9e4b-c0ab9ed393a1 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:13.992514 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481f558-46ba-4445-9e4b-c0ab9ed393a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:13.992990 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481f558-46ba-4445-9e4b-c0ab9ed393a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:14.009292 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:14.097325 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c481f558-46ba-4445-9e4b-c0ab9ed393a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:14.100555 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c481f558-46ba-4445-9e4b-c0ab9ed393a1 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:14.109102 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 190/372] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:24:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:24:14.109967 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc9e644-8d8c-46b0-81d0-2954e8fbf611 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:14.254396 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 183/373] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:24:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:24:14.255389 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c481f558-46ba-4445-9e4b-c0ab9ed393a1 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:14.273051 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc9e644-8d8c-46b0-81d0-2954e8fbf611 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:14.284492 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:14.318152 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:14.325502 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07220acb-4950-44bd-93e1-b2b26ed2cdb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:14.326170 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07220acb-4950-44bd-93e1-b2b26ed2cdb4 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:14.326450 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07220acb-4950-44bd-93e1-b2b26ed2cdb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:14.326725 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07220acb-4950-44bd-93e1-b2b26ed2cdb4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:14.327936 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07220acb-4950-44bd-93e1-b2b26ed2cdb4 None admin] RBAC: Adding query filter params (group.id=66ae82ee5b514025819f0b50ed97d9c9, role.id=aa0552e8ba924f5d9cb1740da7d4a4dc, scope.project.id=cc6e57482c554178aa9b830aa0b83a96) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:14.328250 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07220acb-4950-44bd-93e1-b2b26ed2cdb4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:14.330127 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07220acb-4950-44bd-93e1-b2b26ed2cdb4 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:14.340665 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 191/374] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Jul 18 16:24:14 2022] GET /identity/v3/role_assignments?group.id=66ae82ee5b514025819f0b50ed97d9c9&scope.project.id=cc6e57482c554178aa9b830aa0b83a96&role.id=aa0552e8ba924f5d9cb1740da7d4a4dc => generated 260 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:14.341591 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07220acb-4950-44bd-93e1-b2b26ed2cdb4 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:14.405881 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c481f558-46ba-4445-9e4b-c0ab9ed393a1 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:14.454358 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ca2a92-35b5-4fa3-8769-57324404f3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:14.455656 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca2a92-35b5-4fa3-8769-57324404f3f5 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:14.456228 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca2a92-35b5-4fa3-8769-57324404f3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:14.456775 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca2a92-35b5-4fa3-8769-57324404f3f5 None admin] PATH_INFO: `/v3/projects/9e08d80b293148beac001ab7796d9842` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:14.458242 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ca2a92-35b5-4fa3-8769-57324404f3f5 None admin] RBAC: Authorizing `identity:get_project(project_id=9e08d80b293148beac001ab7796d9842)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:14.459974 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ca2a92-35b5-4fa3-8769-57324404f3f5 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:14.461935 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 184/375] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:24:14 2022] GET /identity/v3/projects/9e08d80b293148beac001ab7796d9842 => generated 344 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:14.468430 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ca2a92-35b5-4fa3-8769-57324404f3f5 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:14.477590 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07220acb-4950-44bd-93e1-b2b26ed2cdb4 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:14.529838 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9caa7d-ade0-4461-8767-77eacf6ac75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:14.532967 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9caa7d-ade0-4461-8767-77eacf6ac75c None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:14.533549 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9caa7d-ade0-4461-8767-77eacf6ac75c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:14.534242 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9caa7d-ade0-4461-8767-77eacf6ac75c None admin] PATH_INFO: `/v3/projects/cc6e57482c554178aa9b830aa0b83a96` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:14.536762 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9caa7d-ade0-4461-8767-77eacf6ac75c None admin] RBAC: Authorizing `identity:get_project(project_id=cc6e57482c554178aa9b830aa0b83a96)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:14.539657 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9caa7d-ade0-4461-8767-77eacf6ac75c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:14.544886 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 192/376] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:24:14 2022] GET /identity/v3/projects/cc6e57482c554178aa9b830aa0b83a96 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:14.551441 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9caa7d-ade0-4461-8767-77eacf6ac75c None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:14.640346 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:14.662121 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ca2a92-35b5-4fa3-8769-57324404f3f5 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:14.688411 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:14.732535 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc15635-b757-40a9-9a11-c1c29e55daea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:14.734009 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc15635-b757-40a9-9a11-c1c29e55daea None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:14.734524 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc15635-b757-40a9-9a11-c1c29e55daea None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:14.735008 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc15635-b757-40a9-9a11-c1c29e55daea None admin] PATH_INFO: `/v3/groups/e7148990fa3f4aa6bf82bc12e49e72cc` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:14.743167 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc15635-b757-40a9-9a11-c1c29e55daea None admin] RBAC: Authorizing `identity:get_group(group_id=e7148990fa3f4aa6bf82bc12e49e72cc)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:14.743698 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9caa7d-ade0-4461-8767-77eacf6ac75c None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:14.745905 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc15635-b757-40a9-9a11-c1c29e55daea None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:14.748412 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 185/377] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jul 18 16:24:14 2022] GET /identity/v3/groups/e7148990fa3f4aa6bf82bc12e49e72cc => generated 229 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:14.750039 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5998d71-8696-410b-a5b3-f04ec41be5e7 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:14.750532 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5998d71-8696-410b-a5b3-f04ec41be5e7 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:14.751045 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5998d71-8696-410b-a5b3-f04ec41be5e7 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:14.753012 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 186/378] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:14.754071 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5998d71-8696-410b-a5b3-f04ec41be5e7 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:14.785196 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48356cf5-d983-46d0-8c0a-d241e0af068b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:14.786137 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48356cf5-d983-46d0-8c0a-d241e0af068b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:14.786418 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48356cf5-d983-46d0-8c0a-d241e0af068b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:14.786697 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48356cf5-d983-46d0-8c0a-d241e0af068b None admin] PATH_INFO: `/v3/groups/66ae82ee5b514025819f0b50ed97d9c9` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:14.788499 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48356cf5-d983-46d0-8c0a-d241e0af068b None admin] RBAC: Authorizing `identity:get_group(group_id=66ae82ee5b514025819f0b50ed97d9c9)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:14.790141 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48356cf5-d983-46d0-8c0a-d241e0af068b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:14.793227 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 193/379] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jul 18 16:24:14 2022] GET /identity/v3/groups/66ae82ee5b514025819f0b50ed97d9c9 => generated 226 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:14.794910 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6471859f-279b-4004-894a-1bf301fdd1ab None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:14.795220 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6471859f-279b-4004-894a-1bf301fdd1ab None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:14.795520 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6471859f-279b-4004-894a-1bf301fdd1ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:14.827584 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-6471859f-279b-4004-894a-1bf301fdd1ab None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:14.829081 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-6471859f-279b-4004-894a-1bf301fdd1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:14.896206 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5998d71-8696-410b-a5b3-f04ec41be5e7 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:14.924433 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 194/380] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:14.925657 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6471859f-279b-4004-894a-1bf301fdd1ab None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:14.948656 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbe2d43-bbed-4de0-849e-01bdaa7fc0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:14.949338 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe2d43-bbed-4de0-849e-01bdaa7fc0a9 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:14.949675 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe2d43-bbed-4de0-849e-01bdaa7fc0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:14.949960 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe2d43-bbed-4de0-849e-01bdaa7fc0a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:14.951055 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbe2d43-bbed-4de0-849e-01bdaa7fc0a9 None admin] RBAC: Adding query filter params (group.id=e7148990fa3f4aa6bf82bc12e49e72cc, role.id=98eaadb3669f4b608d572c001742f559, scope.project.id=9e08d80b293148beac001ab7796d9842) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:14.951856 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbe2d43-bbed-4de0-849e-01bdaa7fc0a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:14.954614 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbe2d43-bbed-4de0-849e-01bdaa7fc0a9 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:14.961442 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 187/381] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Jul 18 16:24:14 2022] GET /identity/v3/role_assignments?group.id=e7148990fa3f4aa6bf82bc12e49e72cc&scope.project.id=9e08d80b293148beac001ab7796d9842&role.id=98eaadb3669f4b608d572c001742f559 => generated 260 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:14.962294 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbe2d43-bbed-4de0-849e-01bdaa7fc0a9 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:14.999088 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:15.018437 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:15.084472 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6471859f-279b-4004-894a-1bf301fdd1ab None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:15.114531 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbe2d43-bbed-4de0-849e-01bdaa7fc0a9 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:15.145913 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a480811-6715-4e90-a259-f3787051e502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:15.147674 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a480811-6715-4e90-a259-f3787051e502 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:15.148441 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a480811-6715-4e90-a259-f3787051e502 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:15.148920 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a480811-6715-4e90-a259-f3787051e502 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:15.151710 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a480811-6715-4e90-a259-f3787051e502 None admin] RBAC: Adding query filter params (group.id=66ae82ee5b514025819f0b50ed97d9c9, role.id=729cd9c0f42d47b6a8eee01e5f5a2d28, scope.project.id=cc6e57482c554178aa9b830aa0b83a96) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:15.152659 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a480811-6715-4e90-a259-f3787051e502 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:15.155041 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a480811-6715-4e90-a259-f3787051e502 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:15.162792 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 195/382] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Jul 18 16:24:14 2022] GET /identity/v3/role_assignments?group.id=66ae82ee5b514025819f0b50ed97d9c9&scope.project.id=cc6e57482c554178aa9b830aa0b83a96&role.id=729cd9c0f42d47b6a8eee01e5f5a2d28 => generated 260 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:15.167650 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6669017-c45d-4453-b699-ba56d90b9b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:15.167650 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6669017-c45d-4453-b699-ba56d90b9b97 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:15.167650 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6669017-c45d-4453-b699-ba56d90b9b97 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:15.167650 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6669017-c45d-4453-b699-ba56d90b9b97 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:15.181260 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6669017-c45d-4453-b699-ba56d90b9b97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:15.189917 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-e6669017-c45d-4453-b699-ba56d90b9b97 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:24:15.189917 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:15.189917 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:15.189917 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:15.189917 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:15.189917 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:15.189917 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:15.189917 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:15.189917 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:15.189917 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:15.189917 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:15.189917 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:15.189917 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:15.189917 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:15.189917 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:15.189917 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:15.190914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:15.190914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:15.190914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:15.190914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:15.190914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:15.190914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:15.190914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:15.190914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:15.190914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:15.190914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:15.190914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:15.190914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:15.190914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:15.190914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:15.190914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:15.190914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:15.191752 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:15.191752 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:15.191752 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:15.191752 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:15.191752 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:15.191752 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:15.191752 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:15.191752 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:15.191752 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:15.191752 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:15.191752 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:24:15.191752 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:15.193107 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 188/383] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 18 16:24:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:15.198939 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a480811-6715-4e90-a259-f3787051e502 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:15.326309 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9b977-d9e6-433d-8dd8-ea82fe8ba6ee None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:15.327179 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9b977-d9e6-433d-8dd8-ea82fe8ba6ee None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:15.327679 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9b977-d9e6-433d-8dd8-ea82fe8ba6ee None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:15.330531 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 189/384] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:15.336459 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a480811-6715-4e90-a259-f3787051e502 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:15.338281 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:15.343466 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce73265-cb3b-4ab4-be09-e8f873f4818b None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:15.344064 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce73265-cb3b-4ab4-be09-e8f873f4818b None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:15.344525 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce73265-cb3b-4ab4-be09-e8f873f4818b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:15.376165 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-bce73265-cb3b-4ab4-be09-e8f873f4818b None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:15.377929 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-bce73265-cb3b-4ab4-be09-e8f873f4818b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:15.398059 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326684e4-4015-433c-aea5-e17842a90653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:15.398780 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326684e4-4015-433c-aea5-e17842a90653 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:15.399053 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326684e4-4015-433c-aea5-e17842a90653 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:15.399330 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326684e4-4015-433c-aea5-e17842a90653 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:15.400175 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326684e4-4015-433c-aea5-e17842a90653 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:15.400520 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326684e4-4015-433c-aea5-e17842a90653 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:15.402209 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326684e4-4015-433c-aea5-e17842a90653 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:15.422662 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 196/385] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:24:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:15.428501 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326684e4-4015-433c-aea5-e17842a90653 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:15.522621 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 190/386] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:24:15.571029 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326684e4-4015-433c-aea5-e17842a90653 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:15.578769 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce73265-cb3b-4ab4-be09-e8f873f4818b None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:15.613227 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfcf3a4-e539-40e7-8d26-ee12e5913bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:15.613227 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcf3a4-e539-40e7-8d26-ee12e5913bd0 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:15.613227 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcf3a4-e539-40e7-8d26-ee12e5913bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:15.613227 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcf3a4-e539-40e7-8d26-ee12e5913bd0 None admin] PATH_INFO: `/v3/groups/66ae82ee5b514025819f0b50ed97d9c9` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:15.614397 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfcf3a4-e539-40e7-8d26-ee12e5913bd0 None admin] RBAC: Authorizing `identity:get_group(group_id=66ae82ee5b514025819f0b50ed97d9c9)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:15.616251 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfcf3a4-e539-40e7-8d26-ee12e5913bd0 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:15.618274 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 197/387] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jul 18 16:24:15 2022] GET /identity/v3/groups/66ae82ee5b514025819f0b50ed97d9c9 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:15.627418 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfcf3a4-e539-40e7-8d26-ee12e5913bd0 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:15.648197 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:15.707891 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:15.782250 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce73265-cb3b-4ab4-be09-e8f873f4818b None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:15.829873 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfcf3a4-e539-40e7-8d26-ee12e5913bd0 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:15.850777 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28fc661-ddb5-428c-b7b7-b01c8c07b454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:15.851634 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28fc661-ddb5-428c-b7b7-b01c8c07b454 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:15.851927 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28fc661-ddb5-428c-b7b7-b01c8c07b454 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:15.852206 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28fc661-ddb5-428c-b7b7-b01c8c07b454 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:15.867725 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28fc661-ddb5-428c-b7b7-b01c8c07b454 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:15.875685 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-b28fc661-ddb5-428c-b7b7-b01c8c07b454 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:24:15.875685 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:15.875685 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:15.875685 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:15.875685 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:15.875685 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:15.875685 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:15.875685 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:15.875685 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:15.875685 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:15.875685 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:15.875685 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:15.875685 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:15.875685 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:15.875685 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:15.875685 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:15.876625 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:15.876625 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:15.876625 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:15.876625 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:15.876625 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:15.876625 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:15.876625 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:15.876625 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:15.876625 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:15.876625 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:15.876625 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:15.876625 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:15.876625 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:15.876625 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:15.876625 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:15.876625 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:15.877421 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:15.877421 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:15.877421 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:15.877421 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:15.877421 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:15.877421 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:15.877421 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:15.877421 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:15.877421 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:15.877421 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:15.877421 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:24:15.877421 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:15.880386 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 191/388] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:24:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:24:15.883366 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28fc661-ddb5-428c-b7b7-b01c8c07b454 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:15.910305 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df29a8-fcd5-4e67-9464-55d1a650393d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:15.911182 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df29a8-fcd5-4e67-9464-55d1a650393d None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:15.911465 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df29a8-fcd5-4e67-9464-55d1a650393d None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:15.911766 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df29a8-fcd5-4e67-9464-55d1a650393d None admin] PATH_INFO: `/v3/projects/04afdb2deaea49078a14a588c9d6de7e` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:15.913180 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7df29a8-fcd5-4e67-9464-55d1a650393d None admin] RBAC: Authorizing `identity:get_project(project_id=04afdb2deaea49078a14a588c9d6de7e)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:15.914740 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7df29a8-fcd5-4e67-9464-55d1a650393d None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:15.916877 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 198/389] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:24:15 2022] GET /identity/v3/projects/04afdb2deaea49078a14a588c9d6de7e => generated 298 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:15.922941 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df29a8-fcd5-4e67-9464-55d1a650393d None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:16.035377 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:16.051142 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:16.055298 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df29a8-fcd5-4e67-9464-55d1a650393d None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:16.065670 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28fc661-ddb5-428c-b7b7-b01c8c07b454 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:16.096960 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f883f9-890d-44ef-b326-4e0f2776c16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:16.097848 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f883f9-890d-44ef-b326-4e0f2776c16f None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:16.098128 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f883f9-890d-44ef-b326-4e0f2776c16f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:16.098474 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f883f9-890d-44ef-b326-4e0f2776c16f None admin] PATH_INFO: `/v3/projects/04afdb2deaea49078a14a588c9d6de7e/groups/66ae82ee5b514025819f0b50ed97d9c9/roles/729cd9c0f42d47b6a8eee01e5f5a2d28` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:16.101474 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f883f9-890d-44ef-b326-4e0f2776c16f None admin] RBAC: Authorizing `identity:create_grant(project_id=04afdb2deaea49078a14a588c9d6de7e, group_id=66ae82ee5b514025819f0b50ed97d9c9, role_id=729cd9c0f42d47b6a8eee01e5f5a2d28)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:16.104220 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f883f9-890d-44ef-b326-4e0f2776c16f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:16.147483 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90286ee5-3a46-4e58-85d5-cc52fa2dc016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:16.148179 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90286ee5-3a46-4e58-85d5-cc52fa2dc016 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:16.148464 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90286ee5-3a46-4e58-85d5-cc52fa2dc016 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:16.148744 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90286ee5-3a46-4e58-85d5-cc52fa2dc016 None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:16.149514 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90286ee5-3a46-4e58-85d5-cc52fa2dc016 None admin] RBAC: Adding query filter params (name=member) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:16.149820 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90286ee5-3a46-4e58-85d5-cc52fa2dc016 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:16.151326 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90286ee5-3a46-4e58-85d5-cc52fa2dc016 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:16.171966 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 192/390] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 18 16:24:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:16.173576 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90286ee5-3a46-4e58-85d5-cc52fa2dc016 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:16.178272 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 199/391] 10.222.0.26 () {58 vars in 1442 bytes} [Mon Jul 18 16:24:15 2022] PUT /identity/v3/projects/04afdb2deaea49078a14a588c9d6de7e/groups/66ae82ee5b514025819f0b50ed97d9c9/roles/729cd9c0f42d47b6a8eee01e5f5a2d28 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:24:16.299010 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90286ee5-3a46-4e58-85d5-cc52fa2dc016 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:16.347217 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571e2143-7c39-41c7-b9bb-95497cdd357d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:16.348151 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571e2143-7c39-41c7-b9bb-95497cdd357d None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:16.348435 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571e2143-7c39-41c7-b9bb-95497cdd357d None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:16.348771 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571e2143-7c39-41c7-b9bb-95497cdd357d None admin] PATH_INFO: `/v3/groups/66ae82ee5b514025819f0b50ed97d9c9` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:16.350120 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571e2143-7c39-41c7-b9bb-95497cdd357d None admin] RBAC: Authorizing `identity:get_group(group_id=66ae82ee5b514025819f0b50ed97d9c9)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:16.351597 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571e2143-7c39-41c7-b9bb-95497cdd357d None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:16.354619 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 193/392] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jul 18 16:24:16 2022] GET /identity/v3/groups/66ae82ee5b514025819f0b50ed97d9c9 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:16.359426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f883f9-890d-44ef-b326-4e0f2776c16f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:16.524464 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f883f9-890d-44ef-b326-4e0f2776c16f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:16.562173 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed3c1e0-1046-4f92-8160-16b21c98c234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:16.563349 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3c1e0-1046-4f92-8160-16b21c98c234 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:16.563869 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3c1e0-1046-4f92-8160-16b21c98c234 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:16.564340 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3c1e0-1046-4f92-8160-16b21c98c234 None admin] PATH_INFO: `/v3/projects/04afdb2deaea49078a14a588c9d6de7e` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:16.565891 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed3c1e0-1046-4f92-8160-16b21c98c234 None admin] RBAC: Authorizing `identity:get_project(project_id=04afdb2deaea49078a14a588c9d6de7e)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:16.567768 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed3c1e0-1046-4f92-8160-16b21c98c234 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:16.569753 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 200/393] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:24:16 2022] GET /identity/v3/projects/04afdb2deaea49078a14a588c9d6de7e => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:16.576432 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571e2143-7c39-41c7-b9bb-95497cdd357d None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:16.605928 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:16.737769 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571e2143-7c39-41c7-b9bb-95497cdd357d None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:16.787916 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5547adb0-6287-4c53-b548-484584496a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:16.788739 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547adb0-6287-4c53-b548-484584496a84 None admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:16.789026 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547adb0-6287-4c53-b548-484584496a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:16.789305 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547adb0-6287-4c53-b548-484584496a84 None admin] PATH_INFO: `/v3/projects/04afdb2deaea49078a14a588c9d6de7e/groups/66ae82ee5b514025819f0b50ed97d9c9/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:16.796914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5547adb0-6287-4c53-b548-484584496a84 None admin] RBAC: Authorizing `identity:create_grant(project_id=04afdb2deaea49078a14a588c9d6de7e, group_id=66ae82ee5b514025819f0b50ed97d9c9, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:16.798377 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5547adb0-6287-4c53-b548-484584496a84 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:16.871196 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 194/394] 10.222.0.26 () {58 vars in 1442 bytes} [Mon Jul 18 16:24:16 2022] PUT /identity/v3/projects/04afdb2deaea49078a14a588c9d6de7e/groups/66ae82ee5b514025819f0b50ed97d9c9/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:24:17.016439 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89045934-40cf-4930-be6f-5a6ebac972a6 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:17.016763 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89045934-40cf-4930-be6f-5a6ebac972a6 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:17.016997 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89045934-40cf-4930-be6f-5a6ebac972a6 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:17.018787 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 201/395] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:17.024322 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c30693-b960-44b3-935c-f506ffcf7ecd None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:17.024614 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c30693-b960-44b3-935c-f506ffcf7ecd None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:17.024896 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c30693-b960-44b3-935c-f506ffcf7ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:17.047545 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-84c30693-b960-44b3-935c-f506ffcf7ecd None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:17.048671 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-84c30693-b960-44b3-935c-f506ffcf7ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:17.157119 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 195/396] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:17.192860 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89045934-40cf-4930-be6f-5a6ebac972a6 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:17.232432 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:17.320287 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89045934-40cf-4930-be6f-5a6ebac972a6 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:17.366274 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f686feb6-aa55-48b1-a5b2-28e194368392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:17.367143 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f686feb6-aa55-48b1-a5b2-28e194368392 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:17.367429 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f686feb6-aa55-48b1-a5b2-28e194368392 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:17.367729 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f686feb6-aa55-48b1-a5b2-28e194368392 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:17.389223 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f686feb6-aa55-48b1-a5b2-28e194368392 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:17.397112 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-f686feb6-aa55-48b1-a5b2-28e194368392 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:24:17.397112 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:17.397112 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:17.397112 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:17.397112 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:17.397112 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:17.397112 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:17.397112 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:17.397112 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:17.397112 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:17.397112 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:17.397112 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:17.397112 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:17.397112 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:17.397112 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:17.397112 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:17.399573 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:17.399573 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:17.399573 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:17.399573 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:17.399573 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:17.399573 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:17.399573 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:17.399573 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:17.399573 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:17.399573 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:17.399573 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:17.399573 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:17.399573 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:17.399573 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:17.399573 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:17.399573 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:17.401789 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:17.401789 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:17.401789 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:17.401789 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:17.401789 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:17.401789 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:17.401789 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:17.401789 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:17.401789 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:17.401789 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:17.401789 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:24:17.401789 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:17.401789 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 202/397] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:24:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:17.408647 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c30693-b960-44b3-935c-f506ffcf7ecd None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:17.428845 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:17.579728 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c30693-b960-44b3-935c-f506ffcf7ecd None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:17.633431 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efe9540-15ca-4397-b1d1-c2a8d7fff03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:17.634116 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe9540-15ca-4397-b1d1-c2a8d7fff03b None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:17.634400 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe9540-15ca-4397-b1d1-c2a8d7fff03b None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:17.634679 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe9540-15ca-4397-b1d1-c2a8d7fff03b None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:17.636169 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efe9540-15ca-4397-b1d1-c2a8d7fff03b None admin] RBAC: Adding query filter params (name=member) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:17.636493 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efe9540-15ca-4397-b1d1-c2a8d7fff03b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:17.637852 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efe9540-15ca-4397-b1d1-c2a8d7fff03b None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:17.652292 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 196/398] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 18 16:24:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:17.658071 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f686feb6-aa55-48b1-a5b2-28e194368392 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:17.735908 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af1147-dcb2-4cc0-8382-695a7028cd62 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:17.736228 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af1147-dcb2-4cc0-8382-695a7028cd62 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:17.736463 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af1147-dcb2-4cc0-8382-695a7028cd62 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:17.738122 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 197/399] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:17.752112 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba784c2-705b-4341-bcfd-8372daaceb9a None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:17.752402 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba784c2-705b-4341-bcfd-8372daaceb9a None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:17.752656 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba784c2-705b-4341-bcfd-8372daaceb9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:17.783064 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:17.784205 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-fba784c2-705b-4341-bcfd-8372daaceb9a None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:17.785426 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-fba784c2-705b-4341-bcfd-8372daaceb9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:17.804231 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f686feb6-aa55-48b1-a5b2-28e194368392 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:17.876493 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230259a5-0106-41a7-a03e-5ccc2db97c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:17.877362 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230259a5-0106-41a7-a03e-5ccc2db97c06 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:17.877647 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230259a5-0106-41a7-a03e-5ccc2db97c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:17.877926 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230259a5-0106-41a7-a03e-5ccc2db97c06 None admin] PATH_INFO: `/v3/groups/66ae82ee5b514025819f0b50ed97d9c9` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:17.880074 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230259a5-0106-41a7-a03e-5ccc2db97c06 None admin] RBAC: Authorizing `identity:get_group(group_id=66ae82ee5b514025819f0b50ed97d9c9)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:17.881586 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230259a5-0106-41a7-a03e-5ccc2db97c06 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:17.884744 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 203/400] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jul 18 16:24:17 2022] GET /identity/v3/groups/66ae82ee5b514025819f0b50ed97d9c9 => generated 226 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:17.891190 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230259a5-0106-41a7-a03e-5ccc2db97c06 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:17.911374 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:17.918739 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 198/401] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:24:17.925413 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28a249-14b3-4485-9997-5b262f699718 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:17.925707 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28a249-14b3-4485-9997-5b262f699718 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:17.931593 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28a249-14b3-4485-9997-5b262f699718 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:17.931593 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 199/402] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:17.935473 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f21c4f-8d11-4b09-aaca-6aec39480441 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:17.935867 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f21c4f-8d11-4b09-aaca-6aec39480441 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:17.936218 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f21c4f-8d11-4b09-aaca-6aec39480441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:17.961472 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-76f21c4f-8d11-4b09-aaca-6aec39480441 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:17.963484 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-76f21c4f-8d11-4b09-aaca-6aec39480441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:18.037372 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230259a5-0106-41a7-a03e-5ccc2db97c06 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:18.037830 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 200/403] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 18 16:24:18.043523 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f21c4f-8d11-4b09-aaca-6aec39480441 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:18.078343 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aa4133-8b9a-4240-9f3d-97646ea9aa1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:18.079242 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa4133-8b9a-4240-9f3d-97646ea9aa1a None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:18.079686 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa4133-8b9a-4240-9f3d-97646ea9aa1a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:18.079960 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa4133-8b9a-4240-9f3d-97646ea9aa1a None admin] PATH_INFO: `/v3/projects/cc6e57482c554178aa9b830aa0b83a96` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:18.081172 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aa4133-8b9a-4240-9f3d-97646ea9aa1a None admin] RBAC: Authorizing `identity:get_project(project_id=cc6e57482c554178aa9b830aa0b83a96)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:18.082731 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aa4133-8b9a-4240-9f3d-97646ea9aa1a None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:18.084960 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 204/404] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:24:17 2022] GET /identity/v3/projects/cc6e57482c554178aa9b830aa0b83a96 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:18.085884 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aa4133-8b9a-4240-9f3d-97646ea9aa1a None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:18.115533 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:18.169566 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f21c4f-8d11-4b09-aaca-6aec39480441 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:18.201504 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aa4133-8b9a-4240-9f3d-97646ea9aa1a None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:18.202529 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:18.208479 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbb147d-594c-4f13-a8e5-5465ec7d5f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:18.209470 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb147d-594c-4f13-a8e5-5465ec7d5f68 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:18.209892 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb147d-594c-4f13-a8e5-5465ec7d5f68 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:18.210302 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb147d-594c-4f13-a8e5-5465ec7d5f68 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:18.222168 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbb147d-594c-4f13-a8e5-5465ec7d5f68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:18.231534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-4fbb147d-594c-4f13-a8e5-5465ec7d5f68 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:24:18.231534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:18.231534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:18.231534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:18.231534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:18.231534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:18.231534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:18.231534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:18.231534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:18.231534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:18.231534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:18.231534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:18.231534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:18.231534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:18.231534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:18.231534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:18.232684 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:18.232684 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:18.232684 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:18.232684 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:18.232684 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:18.232684 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:18.232684 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:18.232684 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:18.232684 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:18.232684 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:18.232684 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:18.232684 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:18.232684 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:18.232684 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:18.232684 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:18.232684 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:18.233622 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:18.233622 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:18.233622 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:18.233622 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:18.233622 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:18.233622 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:18.233622 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:18.233622 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:18.233622 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:18.233622 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:18.233622 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:24:18.233622 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:18.235863 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 201/405] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 18 16:24:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:18.236832 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbb147d-594c-4f13-a8e5-5465ec7d5f68 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:18.252017 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cac11c1-36ed-4ee0-998b-c9dae2f31980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:18.253412 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cac11c1-36ed-4ee0-998b-c9dae2f31980 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:18.253909 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cac11c1-36ed-4ee0-998b-c9dae2f31980 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:18.254377 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cac11c1-36ed-4ee0-998b-c9dae2f31980 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:18.268097 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cac11c1-36ed-4ee0-998b-c9dae2f31980 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:18.277462 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-1cac11c1-36ed-4ee0-998b-c9dae2f31980 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:24:18.277462 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:18.277462 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:18.277462 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:18.277462 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:18.277462 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:18.277462 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:18.277462 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:18.277462 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:18.277462 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:18.277462 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:18.277462 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:18.277462 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:18.277462 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:18.277462 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:18.277462 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:18.278547 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:18.278547 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:18.278547 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:18.278547 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:18.278547 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:18.278547 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:18.278547 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:18.278547 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:18.278547 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:18.278547 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:18.278547 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:18.278547 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:18.278547 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:18.278547 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:18.278547 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:18.278547 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:18.279466 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:18.279466 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:18.279466 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:18.279466 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:18.279466 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:18.279466 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:18.279466 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:18.279466 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:18.279466 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:18.279466 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:18.279466 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:24:18.279466 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:18.283766 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 205/406] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 18 16:24:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:18.284676 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cac11c1-36ed-4ee0-998b-c9dae2f31980 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:18.352578 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbb147d-594c-4f13-a8e5-5465ec7d5f68 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:18.384697 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:18.390399 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0057f2-4bb7-49fa-9cad-d9662987c1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:18.391516 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0057f2-4bb7-49fa-9cad-d9662987c1b0 None admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:18.391940 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0057f2-4bb7-49fa-9cad-d9662987c1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:18.392355 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0057f2-4bb7-49fa-9cad-d9662987c1b0 None admin] PATH_INFO: `/v3/projects/cc6e57482c554178aa9b830aa0b83a96/groups/66ae82ee5b514025819f0b50ed97d9c9/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:18.394569 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0057f2-4bb7-49fa-9cad-d9662987c1b0 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc6e57482c554178aa9b830aa0b83a96, group_id=66ae82ee5b514025819f0b50ed97d9c9, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:18.398067 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0057f2-4bb7-49fa-9cad-d9662987c1b0 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:18.404365 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cac11c1-36ed-4ee0-998b-c9dae2f31980 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:18.443238 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade4494f-b45f-4f80-a857-6087c46951de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:18.443954 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4494f-b45f-4f80-a857-6087c46951de None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:18.444236 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4494f-b45f-4f80-a857-6087c46951de None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:18.444512 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4494f-b45f-4f80-a857-6087c46951de None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:18.445251 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade4494f-b45f-4f80-a857-6087c46951de None admin] RBAC: Adding query filter params (name=admin) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:18.445554 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade4494f-b45f-4f80-a857-6087c46951de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:18.446930 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade4494f-b45f-4f80-a857-6087c46951de None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:18.463712 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 206/407] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:24:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:18.463712 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade4494f-b45f-4f80-a857-6087c46951de None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:18.487148 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 202/408] 10.222.0.26 () {58 vars in 1442 bytes} [Mon Jul 18 16:24:18 2022] PUT /identity/v3/projects/cc6e57482c554178aa9b830aa0b83a96/groups/66ae82ee5b514025819f0b50ed97d9c9/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:24:18.496889 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0057f2-4bb7-49fa-9cad-d9662987c1b0 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:18.613242 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade4494f-b45f-4f80-a857-6087c46951de None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:18.660245 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ce45b6-b555-4518-badc-52ac552b85f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:18.661312 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ce45b6-b555-4518-badc-52ac552b85f7 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:18.661824 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ce45b6-b555-4518-badc-52ac552b85f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:18.662305 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ce45b6-b555-4518-badc-52ac552b85f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:18.663278 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ce45b6-b555-4518-badc-52ac552b85f7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:18.663846 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ce45b6-b555-4518-badc-52ac552b85f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:18.665378 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:18.668537 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ce45b6-b555-4518-badc-52ac552b85f7 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:18.680908 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 207/409] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:24:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:18.687973 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ce45b6-b555-4518-badc-52ac552b85f7 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:18.703278 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:18.753659 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0057f2-4bb7-49fa-9cad-d9662987c1b0 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:18.795909 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c000e1-4a5f-475a-9e6a-e098ee08d5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:18.797124 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c000e1-4a5f-475a-9e6a-e098ee08d5c5 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:18.797124 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c000e1-4a5f-475a-9e6a-e098ee08d5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:18.797379 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c000e1-4a5f-475a-9e6a-e098ee08d5c5 None admin] PATH_INFO: `/v3/groups/e7148990fa3f4aa6bf82bc12e49e72cc` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:18.799680 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c000e1-4a5f-475a-9e6a-e098ee08d5c5 None admin] RBAC: Authorizing `identity:get_group(group_id=e7148990fa3f4aa6bf82bc12e49e72cc)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:18.801211 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c000e1-4a5f-475a-9e6a-e098ee08d5c5 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:18.803406 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 203/410] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jul 18 16:24:18 2022] GET /identity/v3/groups/e7148990fa3f4aa6bf82bc12e49e72cc => generated 229 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:18.810342 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c000e1-4a5f-475a-9e6a-e098ee08d5c5 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:18.827084 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ce45b6-b555-4518-badc-52ac552b85f7 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:18.865538 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2171866b-a672-4463-a455-ba058370ae34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:18.866609 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2171866b-a672-4463-a455-ba058370ae34 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:18.866609 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2171866b-a672-4463-a455-ba058370ae34 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:18.866871 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2171866b-a672-4463-a455-ba058370ae34 None admin] PATH_INFO: `/v3/groups/66ae82ee5b514025819f0b50ed97d9c9` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:18.868131 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2171866b-a672-4463-a455-ba058370ae34 None admin] RBAC: Authorizing `identity:get_group(group_id=66ae82ee5b514025819f0b50ed97d9c9)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:18.870229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2171866b-a672-4463-a455-ba058370ae34 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:18.872225 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 208/411] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jul 18 16:24:18 2022] GET /identity/v3/groups/66ae82ee5b514025819f0b50ed97d9c9 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:18.873455 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87981d8-69b5-446e-af1b-c5cd26d97137 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:18.873746 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87981d8-69b5-446e-af1b-c5cd26d97137 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:18.874026 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87981d8-69b5-446e-af1b-c5cd26d97137 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:18.879862 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 209/412] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:18.884137 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87981d8-69b5-446e-af1b-c5cd26d97137 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:18.948238 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c000e1-4a5f-475a-9e6a-e098ee08d5c5 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:18.965063 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:18.969138 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:18.988765 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0292237-6aaf-451f-8a86-3e7d2e53f9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:18.989888 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0292237-6aaf-451f-8a86-3e7d2e53f9ec None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:18.990322 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0292237-6aaf-451f-8a86-3e7d2e53f9ec None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:18.990745 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0292237-6aaf-451f-8a86-3e7d2e53f9ec None admin] PATH_INFO: `/v3/projects/9e08d80b293148beac001ab7796d9842` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:18.992188 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0292237-6aaf-451f-8a86-3e7d2e53f9ec None admin] RBAC: Authorizing `identity:get_project(project_id=9e08d80b293148beac001ab7796d9842)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:18.992383 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87981d8-69b5-446e-af1b-c5cd26d97137 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:18.993976 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0292237-6aaf-451f-8a86-3e7d2e53f9ec None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:18.997119 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 204/413] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:24:18 2022] GET /identity/v3/projects/9e08d80b293148beac001ab7796d9842 => generated 344 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:18.998606 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239d3a6-071a-4971-9afc-e9358bfeb85a None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:18.999099 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239d3a6-071a-4971-9afc-e9358bfeb85a None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:18.999635 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239d3a6-071a-4971-9afc-e9358bfeb85a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:19.019463 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-d239d3a6-071a-4971-9afc-e9358bfeb85a None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:19.021148 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-d239d3a6-071a-4971-9afc-e9358bfeb85a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:19.031530 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d570250-6d60-417b-b86e-5a15ee2d70c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:19.032403 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d570250-6d60-417b-b86e-5a15ee2d70c1 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:19.032689 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d570250-6d60-417b-b86e-5a15ee2d70c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:19.033018 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d570250-6d60-417b-b86e-5a15ee2d70c1 None admin] PATH_INFO: `/v3/projects/cc6e57482c554178aa9b830aa0b83a96` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:19.034425 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d570250-6d60-417b-b86e-5a15ee2d70c1 None admin] RBAC: Authorizing `identity:get_project(project_id=cc6e57482c554178aa9b830aa0b83a96)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:19.035756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d570250-6d60-417b-b86e-5a15ee2d70c1 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:19.043438 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 210/414] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:24:18 2022] GET /identity/v3/projects/cc6e57482c554178aa9b830aa0b83a96 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:19.044385 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d570250-6d60-417b-b86e-5a15ee2d70c1 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:19.107515 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 205/415] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:19.108883 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d239d3a6-071a-4971-9afc-e9358bfeb85a None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:19.155245 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d570250-6d60-417b-b86e-5a15ee2d70c1 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:19.194492 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5806ed6f-9ff4-4373-8022-21e005e77749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:19.195431 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5806ed6f-9ff4-4373-8022-21e005e77749 None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:19.195765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5806ed6f-9ff4-4373-8022-21e005e77749 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:19.196066 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5806ed6f-9ff4-4373-8022-21e005e77749 None admin] PATH_INFO: `/v3/projects/9e08d80b293148beac001ab7796d9842/groups/e7148990fa3f4aa6bf82bc12e49e72cc/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:19.198281 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5806ed6f-9ff4-4373-8022-21e005e77749 None admin] RBAC: Authorizing `identity:create_grant(project_id=9e08d80b293148beac001ab7796d9842, group_id=e7148990fa3f4aa6bf82bc12e49e72cc, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:19.199769 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5806ed6f-9ff4-4373-8022-21e005e77749 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:19.253579 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d239d3a6-071a-4971-9afc-e9358bfeb85a None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:19.273984 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:19.283011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 211/416] 10.222.0.26 () {58 vars in 1442 bytes} [Mon Jul 18 16:24:19 2022] PUT /identity/v3/projects/9e08d80b293148beac001ab7796d9842/groups/e7148990fa3f4aa6bf82bc12e49e72cc/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:24:19.285191 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5806ed6f-9ff4-4373-8022-21e005e77749 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:19.290938 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:19.308358 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7970c4d-0c3c-4f0d-809a-e8e398ab06ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:19.309466 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7970c4d-0c3c-4f0d-809a-e8e398ab06ba None admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:19.309752 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7970c4d-0c3c-4f0d-809a-e8e398ab06ba None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:19.310033 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7970c4d-0c3c-4f0d-809a-e8e398ab06ba None admin] PATH_INFO: `/v3/projects/cc6e57482c554178aa9b830aa0b83a96/groups/66ae82ee5b514025819f0b50ed97d9c9/roles/729cd9c0f42d47b6a8eee01e5f5a2d28` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:19.316614 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7970c4d-0c3c-4f0d-809a-e8e398ab06ba None admin] RBAC: Authorizing `identity:create_grant(project_id=cc6e57482c554178aa9b830aa0b83a96, group_id=66ae82ee5b514025819f0b50ed97d9c9, role_id=729cd9c0f42d47b6a8eee01e5f5a2d28)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:19.318136 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7970c4d-0c3c-4f0d-809a-e8e398ab06ba None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:19.416814 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 206/417] 10.222.0.26 () {58 vars in 1442 bytes} [Mon Jul 18 16:24:19 2022] PUT /identity/v3/projects/cc6e57482c554178aa9b830aa0b83a96/groups/66ae82ee5b514025819f0b50ed97d9c9/roles/729cd9c0f42d47b6a8eee01e5f5a2d28 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:24:19.428467 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5806ed6f-9ff4-4373-8022-21e005e77749 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:19.457891 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f17c1-4d29-4720-ab1b-e9ff1bfe0515 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:19.458193 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f17c1-4d29-4720-ab1b-e9ff1bfe0515 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:19.458627 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f17c1-4d29-4720-ab1b-e9ff1bfe0515 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:19.461467 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 207/418] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:19.466856 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01078673-3d7f-4ff9-a9aa-a128dc3b9a73 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:19.467150 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01078673-3d7f-4ff9-a9aa-a128dc3b9a73 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:19.467409 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01078673-3d7f-4ff9-a9aa-a128dc3b9a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:19.480257 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6207bd52-d777-4ba2-abdf-8f3b21e40931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:19.481443 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207bd52-d777-4ba2-abdf-8f3b21e40931 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:19.482659 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207bd52-d777-4ba2-abdf-8f3b21e40931 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:19.483153 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207bd52-d777-4ba2-abdf-8f3b21e40931 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:19.488432 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-01078673-3d7f-4ff9-a9aa-a128dc3b9a73 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:19.489681 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-01078673-3d7f-4ff9-a9aa-a128dc3b9a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:19.500508 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6207bd52-d777-4ba2-abdf-8f3b21e40931 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:19.512920 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-6207bd52-d777-4ba2-abdf-8f3b21e40931 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:24:19.512920 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:19.512920 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:19.512920 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:19.512920 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:19.512920 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:19.512920 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:19.512920 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:19.512920 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:19.512920 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:19.512920 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:19.512920 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:19.512920 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:19.512920 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:19.512920 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:19.512920 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:19.513903 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:19.513903 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:19.513903 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:19.513903 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:19.513903 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:19.513903 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:19.513903 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:19.513903 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:19.513903 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:19.513903 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:19.513903 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:19.513903 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:19.513903 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:19.513903 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:19.513903 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:19.513903 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:19.514725 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:19.514725 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:19.514725 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:19.514725 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:19.514725 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:19.514725 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:19.514725 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:19.514725 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:19.514725 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:19.514725 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:19.514725 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:24:19.514725 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:19.518111 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 212/419] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 18 16:24:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:19.524397 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6207bd52-d777-4ba2-abdf-8f3b21e40931 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:19.578278 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 208/420] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:24:19.604707 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01078673-3d7f-4ff9-a9aa-a128dc3b9a73 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:19.608335 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:19.654948 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6207bd52-d777-4ba2-abdf-8f3b21e40931 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:19.707752 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45429807-8ec6-4693-8605-2a6b563528d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:19.708358 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45429807-8ec6-4693-8605-2a6b563528d4 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:19.708648 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45429807-8ec6-4693-8605-2a6b563528d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:19.708923 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45429807-8ec6-4693-8605-2a6b563528d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:19.709683 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45429807-8ec6-4693-8605-2a6b563528d4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:19.709990 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45429807-8ec6-4693-8605-2a6b563528d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:19.712144 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45429807-8ec6-4693-8605-2a6b563528d4 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:19.725742 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 213/421] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:24:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:19.734624 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45429807-8ec6-4693-8605-2a6b563528d4 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:19.747166 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01078673-3d7f-4ff9-a9aa-a128dc3b9a73 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:19.756402 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:19.785899 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e48a63b-0922-4b6f-9503-15936e509b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:19.786736 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e48a63b-0922-4b6f-9503-15936e509b20 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:19.787027 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e48a63b-0922-4b6f-9503-15936e509b20 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:19.787476 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e48a63b-0922-4b6f-9503-15936e509b20 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:19.800680 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e48a63b-0922-4b6f-9503-15936e509b20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:19.810072 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-4e48a63b-0922-4b6f-9503-15936e509b20 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:24:19.810072 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:19.810072 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:19.810072 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:19.810072 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:19.810072 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:19.810072 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:19.810072 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:19.810072 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:19.810072 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:19.810072 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:19.810072 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:19.810072 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:19.810072 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:19.810072 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:19.810072 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:19.811006 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:19.811006 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:19.811006 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:19.811006 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:19.811006 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:19.811006 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:19.811006 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:19.811006 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:19.811006 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:19.811006 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:19.811006 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:19.811006 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:19.811006 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:19.811006 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:19.811006 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:19.811006 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:19.811861 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:19.811861 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:19.811861 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:19.811861 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:19.811861 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:19.811861 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:19.811861 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:19.811861 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:19.811861 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:19.811861 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:19.811861 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:24:19.811861 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:19.811861 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 209/422] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:24:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:24:19.822287 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e48a63b-0922-4b6f-9503-15936e509b20 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:19.862091 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45429807-8ec6-4693-8605-2a6b563528d4 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:19.893763 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:19.903136 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedfe199-d25c-4d9a-ad14-c9ee487c2644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:19.905066 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedfe199-d25c-4d9a-ad14-c9ee487c2644 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:19.905066 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedfe199-d25c-4d9a-ad14-c9ee487c2644 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:19.905066 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedfe199-d25c-4d9a-ad14-c9ee487c2644 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:19.986519 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedfe199-d25c-4d9a-ad14-c9ee487c2644 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:19.988161 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedfe199-d25c-4d9a-ad14-c9ee487c2644 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:20.102002 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 214/423] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:24:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:24:20.110026 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedfe199-d25c-4d9a-ad14-c9ee487c2644 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:20.113875 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e48a63b-0922-4b6f-9503-15936e509b20 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:20.161539 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:20.182069 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f71f74-4466-421c-a28f-9f4fb646ff4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:20.182818 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f71f74-4466-421c-a28f-9f4fb646ff4c None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:20.183114 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f71f74-4466-421c-a28f-9f4fb646ff4c None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:20.183396 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f71f74-4466-421c-a28f-9f4fb646ff4c None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:20.184190 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f71f74-4466-421c-a28f-9f4fb646ff4c None admin] RBAC: Adding query filter params (name=member) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:20.184501 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f71f74-4466-421c-a28f-9f4fb646ff4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:20.185893 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f71f74-4466-421c-a28f-9f4fb646ff4c None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:20.205203 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 210/424] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 18 16:24:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:20.209431 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f71f74-4466-421c-a28f-9f4fb646ff4c None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:20.255156 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:20.331537 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedfe199-d25c-4d9a-ad14-c9ee487c2644 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:20.340716 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f71f74-4466-421c-a28f-9f4fb646ff4c None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:20.383003 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7dd654-5244-43d7-bfbc-c1e3b1708dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:20.384017 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7dd654-5244-43d7-bfbc-c1e3b1708dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:20.386072 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7dd654-5244-43d7-bfbc-c1e3b1708dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:20.386557 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7dd654-5244-43d7-bfbc-c1e3b1708dd9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:20.415029 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5000f72f-8ef4-455f-9013-53828b8e1d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:20.417141 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000f72f-8ef4-455f-9013-53828b8e1d63 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:20.418879 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000f72f-8ef4-455f-9013-53828b8e1d63 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:20.419360 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000f72f-8ef4-455f-9013-53828b8e1d63 None admin] PATH_INFO: `/v3/projects/04afdb2deaea49078a14a588c9d6de7e` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:20.420959 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5000f72f-8ef4-455f-9013-53828b8e1d63 None admin] RBAC: Authorizing `identity:get_project(project_id=04afdb2deaea49078a14a588c9d6de7e)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:20.425664 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5000f72f-8ef4-455f-9013-53828b8e1d63 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:20.428837 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 215/425] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:24:20 2022] GET /identity/v3/projects/04afdb2deaea49078a14a588c9d6de7e => generated 298 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:20.435714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5000f72f-8ef4-455f-9013-53828b8e1d63 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:20.502986 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7dd654-5244-43d7-bfbc-c1e3b1708dd9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:20.505025 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7dd654-5244-43d7-bfbc-c1e3b1708dd9 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:20.540967 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:20.626972 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 211/426] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:24:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:24:20.636615 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7dd654-5244-43d7-bfbc-c1e3b1708dd9 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:20.686295 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:20.702773 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5000f72f-8ef4-455f-9013-53828b8e1d63 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:20.748399 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7dd654-5244-43d7-bfbc-c1e3b1708dd9 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:20.763724 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51cdbf6-5ab7-416e-b0df-4d6090baad97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:20.764592 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51cdbf6-5ab7-416e-b0df-4d6090baad97 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:20.764873 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51cdbf6-5ab7-416e-b0df-4d6090baad97 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:20.765149 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51cdbf6-5ab7-416e-b0df-4d6090baad97 None admin] PATH_INFO: `/v3/groups/66ae82ee5b514025819f0b50ed97d9c9` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:20.766408 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51cdbf6-5ab7-416e-b0df-4d6090baad97 None admin] RBAC: Authorizing `identity:get_group(group_id=66ae82ee5b514025819f0b50ed97d9c9)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:20.767998 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51cdbf6-5ab7-416e-b0df-4d6090baad97 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:20.770045 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 216/427] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jul 18 16:24:20 2022] GET /identity/v3/groups/66ae82ee5b514025819f0b50ed97d9c9 => generated 226 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:20.776659 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51cdbf6-5ab7-416e-b0df-4d6090baad97 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:20.778007 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:20.801755 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05023897-940d-4e54-9c78-995e9753f9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:20.802608 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05023897-940d-4e54-9c78-995e9753f9e8 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:20.802835 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05023897-940d-4e54-9c78-995e9753f9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:20.803111 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05023897-940d-4e54-9c78-995e9753f9e8 None admin] PATH_INFO: `/v3/projects/04afdb2deaea49078a14a588c9d6de7e` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:20.809367 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05023897-940d-4e54-9c78-995e9753f9e8 None admin] RBAC: Authorizing `identity:get_project(project_id=04afdb2deaea49078a14a588c9d6de7e)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:20.810584 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05023897-940d-4e54-9c78-995e9753f9e8 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:20.812904 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 212/428] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:24:20 2022] GET /identity/v3/projects/04afdb2deaea49078a14a588c9d6de7e => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:20.818262 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05023897-940d-4e54-9c78-995e9753f9e8 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:20.909894 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51cdbf6-5ab7-416e-b0df-4d6090baad97 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:20.947358 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6d8469-5bf6-4310-a525-c0411548f527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:20.950972 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d8469-5bf6-4310-a525-c0411548f527 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:20.951484 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d8469-5bf6-4310-a525-c0411548f527 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:20.951980 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d8469-5bf6-4310-a525-c0411548f527 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:20.953030 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6d8469-5bf6-4310-a525-c0411548f527 None admin] RBAC: Adding query filter params (group.id=66ae82ee5b514025819f0b50ed97d9c9, role.id=729cd9c0f42d47b6a8eee01e5f5a2d28, scope.project.id=04afdb2deaea49078a14a588c9d6de7e) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:20.953541 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6d8469-5bf6-4310-a525-c0411548f527 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:20.955461 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6d8469-5bf6-4310-a525-c0411548f527 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:20.962863 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 217/429] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Jul 18 16:24:20 2022] GET /identity/v3/role_assignments?group.id=66ae82ee5b514025819f0b50ed97d9c9&scope.project.id=04afdb2deaea49078a14a588c9d6de7e&role.id=729cd9c0f42d47b6a8eee01e5f5a2d28 => generated 611 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:21.040290 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05023897-940d-4e54-9c78-995e9753f9e8 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:21.060544 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47275c48-2513-4cd2-8226-f5387a23c361 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:21.061258 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47275c48-2513-4cd2-8226-f5387a23c361 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:21.061862 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47275c48-2513-4cd2-8226-f5387a23c361 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:21.067970 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 218/430] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:21 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:21.073648 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b55007-1c4c-4631-aa18-db294e12244a None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:21.074068 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b55007-1c4c-4631-aa18-db294e12244a None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:21.074496 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b55007-1c4c-4631-aa18-db294e12244a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:21.082275 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425d2a14-83a0-44f9-bbda-88456cbdda0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:21.083374 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d2a14-83a0-44f9-bbda-88456cbdda0f None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:21.083828 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d2a14-83a0-44f9-bbda-88456cbdda0f None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:21.084262 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d2a14-83a0-44f9-bbda-88456cbdda0f None admin] PATH_INFO: `/v3/groups/66ae82ee5b514025819f0b50ed97d9c9` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:21.085639 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425d2a14-83a0-44f9-bbda-88456cbdda0f None admin] RBAC: Authorizing `identity:get_group(group_id=66ae82ee5b514025819f0b50ed97d9c9)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:21.087260 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425d2a14-83a0-44f9-bbda-88456cbdda0f None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:21.090199 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 213/431] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jul 18 16:24:20 2022] GET /identity/v3/groups/66ae82ee5b514025819f0b50ed97d9c9 => generated 226 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:21.096018 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425d2a14-83a0-44f9-bbda-88456cbdda0f None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:21.103188 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-09b55007-1c4c-4631-aa18-db294e12244a None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:21.105487 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-09b55007-1c4c-4631-aa18-db294e12244a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:21.152557 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:21.202206 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:21.225543 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 219/432] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:24:21.254143 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b55007-1c4c-4631-aa18-db294e12244a None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:21.267178 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425d2a14-83a0-44f9-bbda-88456cbdda0f None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:21.304932 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416968b2-4494-41b4-8eac-b0e4e49c05eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:21.305964 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416968b2-4494-41b4-8eac-b0e4e49c05eb None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:21.306450 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416968b2-4494-41b4-8eac-b0e4e49c05eb None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:21.307567 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416968b2-4494-41b4-8eac-b0e4e49c05eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:21.308617 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416968b2-4494-41b4-8eac-b0e4e49c05eb None admin] RBAC: Adding query filter params (group.id=66ae82ee5b514025819f0b50ed97d9c9, role.id=aa0552e8ba924f5d9cb1740da7d4a4dc, scope.project.id=04afdb2deaea49078a14a588c9d6de7e) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:21.309126 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416968b2-4494-41b4-8eac-b0e4e49c05eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:21.310842 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416968b2-4494-41b4-8eac-b0e4e49c05eb None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:21.323650 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 214/433] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Jul 18 16:24:21 2022] GET /identity/v3/role_assignments?group.id=66ae82ee5b514025819f0b50ed97d9c9&scope.project.id=04afdb2deaea49078a14a588c9d6de7e&role.id=aa0552e8ba924f5d9cb1740da7d4a4dc => generated 611 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:21.386004 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b55007-1c4c-4631-aa18-db294e12244a None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:21.430233 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5155748-7d80-4f89-bca0-a2cac7a12561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:21.431068 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5155748-7d80-4f89-bca0-a2cac7a12561 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:21.431352 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5155748-7d80-4f89-bca0-a2cac7a12561 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:21.431665 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5155748-7d80-4f89-bca0-a2cac7a12561 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:21.447803 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5155748-7d80-4f89-bca0-a2cac7a12561 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:21.456040 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-b5155748-7d80-4f89-bca0-a2cac7a12561 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:24:21.456040 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:21.456040 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:21.456040 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:21.456040 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:21.456040 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:21.456040 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:21.456040 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:21.456040 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:21.456040 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:21.456040 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:21.456040 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:21.456040 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:21.456040 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:21.456040 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:21.456040 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:21.457025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:21.457025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:21.457025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:21.457025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:21.457025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:21.457025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:21.457025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:21.457025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:21.457025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:21.457025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:21.457025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:21.457025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:21.457025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:21.457025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:21.457025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:21.457025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:21.457850 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:21.457850 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:21.457850 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:21.457850 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:21.457850 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:21.457850 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:21.457850 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:21.457850 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:21.457850 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:21.457850 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:21.457850 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:24:21.457850 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:21.457850 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 220/434] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:24:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:24:21.467166 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416968b2-4494-41b4-8eac-b0e4e49c05eb None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:21.587787 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416968b2-4494-41b4-8eac-b0e4e49c05eb None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:21.624298 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:21.632264 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9433dd8-6f49-45f7-8d37-02edc6284e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:21.632956 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9433dd8-6f49-45f7-8d37-02edc6284e33 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:21.633244 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9433dd8-6f49-45f7-8d37-02edc6284e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:21.633526 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9433dd8-6f49-45f7-8d37-02edc6284e33 None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:21.634295 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9433dd8-6f49-45f7-8d37-02edc6284e33 None admin] RBAC: Adding query filter params (name=member) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:21.634603 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9433dd8-6f49-45f7-8d37-02edc6284e33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:21.636651 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9433dd8-6f49-45f7-8d37-02edc6284e33 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:21.646550 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 215/435] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 18 16:24:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:21.653075 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5155748-7d80-4f89-bca0-a2cac7a12561 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:21.677798 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:21.770137 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5155748-7d80-4f89-bca0-a2cac7a12561 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:21.811185 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1101986b-4ddd-423b-8392-fbb0636520e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:21.812254 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1101986b-4ddd-423b-8392-fbb0636520e4 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:21.812734 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1101986b-4ddd-423b-8392-fbb0636520e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:21.813236 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1101986b-4ddd-423b-8392-fbb0636520e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:21.844189 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be096548-9194-4915-b93c-a3e3766f2600 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:21.844189 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be096548-9194-4915-b93c-a3e3766f2600 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:21.844189 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be096548-9194-4915-b93c-a3e3766f2600 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:21.844767 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 216/436] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:21.850362 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ee49b-924c-4478-80b7-af4057756733 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:21.850640 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ee49b-924c-4478-80b7-af4057756733 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:21.852156 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ee49b-924c-4478-80b7-af4057756733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:21.872609 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-e44ee49b-924c-4478-80b7-af4057756733 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:21.874171 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-e44ee49b-924c-4478-80b7-af4057756733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:21.899688 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1101986b-4ddd-423b-8392-fbb0636520e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:21.902136 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1101986b-4ddd-423b-8392-fbb0636520e4 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:21.925455 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:21.971259 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 217/437] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:24:21.998147 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44ee49b-924c-4478-80b7-af4057756733 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:22.025548 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 221/438] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:24:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:24:22.031372 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1101986b-4ddd-423b-8392-fbb0636520e4 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:22.115021 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44ee49b-924c-4478-80b7-af4057756733 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:22.135524 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:22.142636 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1101986b-4ddd-423b-8392-fbb0636520e4 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:22.152319 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb16492c-0114-4b1d-b501-9127a0b83f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:22.153137 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb16492c-0114-4b1d-b501-9127a0b83f27 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:22.153419 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb16492c-0114-4b1d-b501-9127a0b83f27 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:22.153699 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb16492c-0114-4b1d-b501-9127a0b83f27 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:22.160148 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:22.164323 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb16492c-0114-4b1d-b501-9127a0b83f27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:22.173618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-bb16492c-0114-4b1d-b501-9127a0b83f27 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:24:22.173618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:22.173618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:22.173618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:22.173618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:22.173618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:22.173618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:22.173618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:22.173618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:22.173618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:22.173618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:22.173618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:22.173618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:22.173618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:22.173618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:22.173618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:22.174729 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:22.174729 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:22.174729 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:22.174729 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:22.174729 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:22.174729 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:22.174729 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:22.174729 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:22.174729 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:22.174729 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:22.174729 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:22.174729 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:22.174729 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:22.174729 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:22.174729 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:22.174729 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:22.175762 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:22.175762 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:22.175762 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:22.175762 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:22.175762 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:22.175762 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:22.175762 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:22.175762 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:22.175762 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:22.175762 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:22.175762 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:24:22.175762 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:22.178017 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 218/439] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 18 16:24:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:24:22.178629 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f8696c-ba46-4349-b16f-835694ba13ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:22.179283 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf63adfb-d76c-44b9-9c48-b351702b7580 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:22.179601 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f8696c-ba46-4349-b16f-835694ba13ca None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:22.179892 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f8696c-ba46-4349-b16f-835694ba13ca None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:22.184785 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f8696c-ba46-4349-b16f-835694ba13ca None admin] PATH_INFO: `/v3/projects/cc6e57482c554178aa9b830aa0b83a96` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:22.184897 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf63adfb-d76c-44b9-9c48-b351702b7580 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:22.184897 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf63adfb-d76c-44b9-9c48-b351702b7580 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:22.185024 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f8696c-ba46-4349-b16f-835694ba13ca None admin] RBAC: Authorizing `identity:get_project(project_id=cc6e57482c554178aa9b830aa0b83a96)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:22.185024 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f8696c-ba46-4349-b16f-835694ba13ca None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:22.185172 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 219/440] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:22.185715 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf63adfb-d76c-44b9-9c48-b351702b7580 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:22.189097 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 222/441] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:24:22 2022] GET /identity/v3/projects/cc6e57482c554178aa9b830aa0b83a96 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:22.189097 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91124f18-b499-4035-810b-bbe29f4d97dd None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:22.189097 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91124f18-b499-4035-810b-bbe29f4d97dd None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:22.189097 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91124f18-b499-4035-810b-bbe29f4d97dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:22.206513 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-91124f18-b499-4035-810b-bbe29f4d97dd None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:22.207764 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-91124f18-b499-4035-810b-bbe29f4d97dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:22.277696 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 223/442] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:24:22.278553 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91124f18-b499-4035-810b-bbe29f4d97dd None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:22.295996 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf63adfb-d76c-44b9-9c48-b351702b7580 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:22.332097 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1513797-9dfd-4e76-b7e3-2dbd7e0fb053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:22.332775 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1513797-9dfd-4e76-b7e3-2dbd7e0fb053 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:22.333060 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1513797-9dfd-4e76-b7e3-2dbd7e0fb053 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:22.333339 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1513797-9dfd-4e76-b7e3-2dbd7e0fb053 None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:22.334093 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1513797-9dfd-4e76-b7e3-2dbd7e0fb053 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:22.334404 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1513797-9dfd-4e76-b7e3-2dbd7e0fb053 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:22.335831 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1513797-9dfd-4e76-b7e3-2dbd7e0fb053 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:22.345814 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 220/443] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:24:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:22.346649 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1513797-9dfd-4e76-b7e3-2dbd7e0fb053 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:22.388261 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91124f18-b499-4035-810b-bbe29f4d97dd None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:22.393096 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:22.403279 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:22.423788 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd38e8c-e158-425f-9a9a-2cb1506133fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:22.424667 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd38e8c-e158-425f-9a9a-2cb1506133fe None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:22.424950 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd38e8c-e158-425f-9a9a-2cb1506133fe None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:22.425268 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd38e8c-e158-425f-9a9a-2cb1506133fe None admin] PATH_INFO: `/v3/groups/66ae82ee5b514025819f0b50ed97d9c9` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:22.426467 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd38e8c-e158-425f-9a9a-2cb1506133fe None admin] RBAC: Authorizing `identity:get_group(group_id=66ae82ee5b514025819f0b50ed97d9c9)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:22.427947 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd38e8c-e158-425f-9a9a-2cb1506133fe None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:22.429761 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 224/444] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jul 18 16:24:22 2022] GET /identity/v3/groups/66ae82ee5b514025819f0b50ed97d9c9 => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:22.430424 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd38e8c-e158-425f-9a9a-2cb1506133fe None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:22.457687 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1513797-9dfd-4e76-b7e3-2dbd7e0fb053 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:22.492636 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe0c2c5-f261-40af-939c-ea601d650630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:22.493452 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe0c2c5-f261-40af-939c-ea601d650630 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:22.493733 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe0c2c5-f261-40af-939c-ea601d650630 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:22.494006 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe0c2c5-f261-40af-939c-ea601d650630 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:22.504224 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe0c2c5-f261-40af-939c-ea601d650630 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:22.512438 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-dfe0c2c5-f261-40af-939c-ea601d650630 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:24:22.512438 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:22.512438 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:22.512438 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:22.512438 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:22.512438 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:22.512438 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:22.512438 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:22.512438 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:22.512438 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:22.512438 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:22.512438 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:22.512438 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:22.512438 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:22.512438 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:22.512438 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:22.513408 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:22.513408 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:22.513408 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:22.513408 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:22.513408 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:22.513408 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:22.513408 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:22.513408 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:22.513408 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:22.513408 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:22.513408 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:22.513408 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:22.513408 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:22.513408 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:22.513408 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:22.513408 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:22.514204 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:22.514204 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:22.514204 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:22.514204 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:22.514204 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:22.514204 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:22.514204 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:22.514204 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:22.514204 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:22.514204 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:22.514204 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:24:22.514204 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:22.514204 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 221/445] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 18 16:24:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:22.514945 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe0c2c5-f261-40af-939c-ea601d650630 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:22.539360 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd38e8c-e158-425f-9a9a-2cb1506133fe None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:22.573674 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb500744-40ad-47d2-9c98-bc6a20898a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:22.574323 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb500744-40ad-47d2-9c98-bc6a20898a55 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:22.574603 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb500744-40ad-47d2-9c98-bc6a20898a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:22.574883 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb500744-40ad-47d2-9c98-bc6a20898a55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:22.619486 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe0c2c5-f261-40af-939c-ea601d650630 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:22.621723 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:22.624729 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:22.644168 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb500744-40ad-47d2-9c98-bc6a20898a55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:22.645568 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb500744-40ad-47d2-9c98-bc6a20898a55 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:22.657125 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aba90f-aadc-4a7a-a897-9588a03bf7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:22.657791 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aba90f-aadc-4a7a-a897-9588a03bf7d1 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:22.658078 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aba90f-aadc-4a7a-a897-9588a03bf7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:22.658355 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aba90f-aadc-4a7a-a897-9588a03bf7d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:22.659203 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aba90f-aadc-4a7a-a897-9588a03bf7d1 None admin] RBAC: Adding query filter params (group.id=66ae82ee5b514025819f0b50ed97d9c9, role.id=aa0552e8ba924f5d9cb1740da7d4a4dc, scope.project.id=cc6e57482c554178aa9b830aa0b83a96) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:22.659571 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aba90f-aadc-4a7a-a897-9588a03bf7d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:22.661170 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aba90f-aadc-4a7a-a897-9588a03bf7d1 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:22.668513 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 222/446] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Jul 18 16:24:22 2022] GET /identity/v3/role_assignments?group.id=66ae82ee5b514025819f0b50ed97d9c9&scope.project.id=cc6e57482c554178aa9b830aa0b83a96&role.id=aa0552e8ba924f5d9cb1740da7d4a4dc => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:22.669355 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aba90f-aadc-4a7a-a897-9588a03bf7d1 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:22.768813 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 225/447] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:24:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:24:22.774456 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb500744-40ad-47d2-9c98-bc6a20898a55 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:22.793376 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aba90f-aadc-4a7a-a897-9588a03bf7d1 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:22.844077 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20fa74a-9300-457e-8561-9eb0db153ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:22.845498 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20fa74a-9300-457e-8561-9eb0db153ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:22.845794 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20fa74a-9300-457e-8561-9eb0db153ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:22.846068 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20fa74a-9300-457e-8561-9eb0db153ae8 None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:22.846848 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20fa74a-9300-457e-8561-9eb0db153ae8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:22.847154 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20fa74a-9300-457e-8561-9eb0db153ae8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:22.848607 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20fa74a-9300-457e-8561-9eb0db153ae8 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:22.859877 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 223/448] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:24:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:22.866266 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20fa74a-9300-457e-8561-9eb0db153ae8 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:22.873198 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:22.892988 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb500744-40ad-47d2-9c98-bc6a20898a55 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:22.900153 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:22.939368 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b395f7fc-34d9-4a6b-b178-05a1907d1021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:22.940260 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395f7fc-34d9-4a6b-b178-05a1907d1021 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:22.941860 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395f7fc-34d9-4a6b-b178-05a1907d1021 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:22.941860 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395f7fc-34d9-4a6b-b178-05a1907d1021 None admin] PATH_INFO: `/v3/projects/9e08d80b293148beac001ab7796d9842` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:22.942120 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b395f7fc-34d9-4a6b-b178-05a1907d1021 None admin] RBAC: Authorizing `identity:get_project(project_id=9e08d80b293148beac001ab7796d9842)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:22.944520 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b395f7fc-34d9-4a6b-b178-05a1907d1021 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:22.945319 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 226/449] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:24:22 2022] GET /identity/v3/projects/9e08d80b293148beac001ab7796d9842 => generated 344 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:22.952887 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b395f7fc-34d9-4a6b-b178-05a1907d1021 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:23.003751 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20fa74a-9300-457e-8561-9eb0db153ae8 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:23.063903 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e358e-4b8f-48b3-8691-a03bfc8e4041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:23.064930 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e358e-4b8f-48b3-8691-a03bfc8e4041 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:23.065418 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e358e-4b8f-48b3-8691-a03bfc8e4041 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:23.065892 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e358e-4b8f-48b3-8691-a03bfc8e4041 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:23.091008 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b395f7fc-34d9-4a6b-b178-05a1907d1021 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:23.138987 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680abecd-4cab-40a7-b2c6-b06e70853b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:23.139902 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680abecd-4cab-40a7-b2c6-b06e70853b80 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:23.140190 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680abecd-4cab-40a7-b2c6-b06e70853b80 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:23.140469 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680abecd-4cab-40a7-b2c6-b06e70853b80 None admin] PATH_INFO: `/v3/groups/e7148990fa3f4aa6bf82bc12e49e72cc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:23.141680 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680abecd-4cab-40a7-b2c6-b06e70853b80 None admin] RBAC: Authorizing `identity:get_group(group_id=e7148990fa3f4aa6bf82bc12e49e72cc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:23.143128 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680abecd-4cab-40a7-b2c6-b06e70853b80 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:23.144809 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 227/450] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jul 18 16:24:22 2022] GET /identity/v3/groups/e7148990fa3f4aa6bf82bc12e49e72cc => generated 229 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:23.151460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680abecd-4cab-40a7-b2c6-b06e70853b80 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:23.153932 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3e358e-4b8f-48b3-8691-a03bfc8e4041 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:23.155426 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3e358e-4b8f-48b3-8691-a03bfc8e4041 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:23.170951 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:23.183750 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:23.251540 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680abecd-4cab-40a7-b2c6-b06e70853b80 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:23.257215 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 224/451] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:24:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:24:23.262932 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e358e-4b8f-48b3-8691-a03bfc8e4041 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:23.286551 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b13c64a-b276-4a7a-9f45-c0e25112a448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:23.287208 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13c64a-b276-4a7a-9f45-c0e25112a448 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:23.287523 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13c64a-b276-4a7a-9f45-c0e25112a448 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:23.287794 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13c64a-b276-4a7a-9f45-c0e25112a448 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:23.288630 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b13c64a-b276-4a7a-9f45-c0e25112a448 None admin] RBAC: Adding query filter params (group.id=e7148990fa3f4aa6bf82bc12e49e72cc, role.id=98eaadb3669f4b608d572c001742f559, scope.project.id=9e08d80b293148beac001ab7796d9842) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:23.288944 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b13c64a-b276-4a7a-9f45-c0e25112a448 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:23.290382 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b13c64a-b276-4a7a-9f45-c0e25112a448 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:23.297515 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 228/452] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Jul 18 16:24:23 2022] GET /identity/v3/role_assignments?group.id=e7148990fa3f4aa6bf82bc12e49e72cc&scope.project.id=9e08d80b293148beac001ab7796d9842&role.id=98eaadb3669f4b608d572c001742f559 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:23.372826 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e358e-4b8f-48b3-8691-a03bfc8e4041 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:23.406219 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:23.414283 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf5c1ec-c840-45e6-83c4-a4460f2558d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:23.415172 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf5c1ec-c840-45e6-83c4-a4460f2558d7 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:23.415453 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf5c1ec-c840-45e6-83c4-a4460f2558d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:23.415771 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf5c1ec-c840-45e6-83c4-a4460f2558d7 None admin] PATH_INFO: `/v3/projects/cc6e57482c554178aa9b830aa0b83a96` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:23.419422 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf5c1ec-c840-45e6-83c4-a4460f2558d7 None admin] RBAC: Authorizing `identity:get_project(project_id=cc6e57482c554178aa9b830aa0b83a96)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:23.419422 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf5c1ec-c840-45e6-83c4-a4460f2558d7 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:23.420834 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 225/453] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:24:23 2022] GET /identity/v3/projects/cc6e57482c554178aa9b830aa0b83a96 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:23.426512 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b13c64a-b276-4a7a-9f45-c0e25112a448 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:23.520436 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:23.538227 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b13c64a-b276-4a7a-9f45-c0e25112a448 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:23.571706 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefa8723-7f04-44e4-93c2-1f54168bcbdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:23.573050 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa8723-7f04-44e4-93c2-1f54168bcbdd None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:23.573339 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa8723-7f04-44e4-93c2-1f54168bcbdd None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:23.573609 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa8723-7f04-44e4-93c2-1f54168bcbdd None admin] PATH_INFO: `/v3/groups/66ae82ee5b514025819f0b50ed97d9c9` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:23.574793 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefa8723-7f04-44e4-93c2-1f54168bcbdd None admin] RBAC: Authorizing `identity:get_group(group_id=66ae82ee5b514025819f0b50ed97d9c9)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:23.576283 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefa8723-7f04-44e4-93c2-1f54168bcbdd None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:23.583153 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 229/454] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jul 18 16:24:23 2022] GET /identity/v3/groups/66ae82ee5b514025819f0b50ed97d9c9 => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:23.583883 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf5c1ec-c840-45e6-83c4-a4460f2558d7 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:23.685842 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf5c1ec-c840-45e6-83c4-a4460f2558d7 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:23.719449 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90871496-60a8-42b2-af34-0caeda088bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:23.720146 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90871496-60a8-42b2-af34-0caeda088bf4 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:23.720431 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90871496-60a8-42b2-af34-0caeda088bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:23.720710 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90871496-60a8-42b2-af34-0caeda088bf4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:23.721556 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90871496-60a8-42b2-af34-0caeda088bf4 None admin] RBAC: Adding query filter params (group.id=66ae82ee5b514025819f0b50ed97d9c9, role.id=729cd9c0f42d47b6a8eee01e5f5a2d28, scope.project.id=cc6e57482c554178aa9b830aa0b83a96) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:23.721876 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90871496-60a8-42b2-af34-0caeda088bf4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:23.723331 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90871496-60a8-42b2-af34-0caeda088bf4 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:23.729770 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 226/455] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Jul 18 16:24:23 2022] GET /identity/v3/role_assignments?group.id=66ae82ee5b514025819f0b50ed97d9c9&scope.project.id=cc6e57482c554178aa9b830aa0b83a96&role.id=729cd9c0f42d47b6a8eee01e5f5a2d28 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:26.397833 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6cc3cb-ca0e-4a01-8bd3-781b36e163f9 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:26.398342 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6cc3cb-ca0e-4a01-8bd3-781b36e163f9 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:26.398415 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6cc3cb-ca0e-4a01-8bd3-781b36e163f9 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:26.400136 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 230/456] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:26.415434 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a272d83-c9d5-4e79-940c-4996635b8936 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:26.415434 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a272d83-c9d5-4e79-940c-4996635b8936 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:26.415434 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a272d83-c9d5-4e79-940c-4996635b8936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:26.440692 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-4a272d83-c9d5-4e79-940c-4996635b8936 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:26.441877 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-4a272d83-c9d5-4e79-940c-4996635b8936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:26.583942 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:26.614495 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a0641e-ff98-4c4e-9541-b3441ba46f10 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:26.615130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a0641e-ff98-4c4e-9541-b3441ba46f10 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:26.615653 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a0641e-ff98-4c4e-9541-b3441ba46f10 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:26.617000 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 227/457] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:26.618190 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 231/458] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:26.638838 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c42c0-0c9f-45bc-b9c2-60b14f9654cb None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:26.639142 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c42c0-0c9f-45bc-b9c2-60b14f9654cb None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:26.639394 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c42c0-0c9f-45bc-b9c2-60b14f9654cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:26.653624 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a0641e-ff98-4c4e-9541-b3441ba46f10 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:26.663373 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-8c8c42c0-0c9f-45bc-b9c2-60b14f9654cb None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:26.664609 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-8c8c42c0-0c9f-45bc-b9c2-60b14f9654cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:26.835607 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 228/459] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:26.856294 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219dd3e7-0edc-47fb-b4d6-5e9cbc5002e0 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:26.856599 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219dd3e7-0edc-47fb-b4d6-5e9cbc5002e0 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:26.856851 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219dd3e7-0edc-47fb-b4d6-5e9cbc5002e0 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:26.858554 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 229/460] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:24:26.863464 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6707f-28e7-4e71-9e6d-04e22252bce0 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:26.863779 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6707f-28e7-4e71-9e6d-04e22252bce0 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:26.864057 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6707f-28e7-4e71-9e6d-04e22252bce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:26.887418 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-2ad6707f-28e7-4e71-9e6d-04e22252bce0 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:26.888912 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-2ad6707f-28e7-4e71-9e6d-04e22252bce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:26.895722 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a0641e-ff98-4c4e-9541-b3441ba46f10 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:26.936736 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2042f256-2c00-4ffa-917b-02c921e9e003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:26.937777 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042f256-2c00-4ffa-917b-02c921e9e003 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:26.938464 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042f256-2c00-4ffa-917b-02c921e9e003 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:26.939125 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042f256-2c00-4ffa-917b-02c921e9e003 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:26.951189 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2042f256-2c00-4ffa-917b-02c921e9e003 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:26.952918 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2042f256-2c00-4ffa-917b-02c921e9e003 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:26.968726 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 230/461] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:24:26.969597 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad6707f-28e7-4e71-9e6d-04e22252bce0 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:26.972293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-2042f256-2c00-4ffa-917b-02c921e9e003 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:26.972293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:26.972293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:26.972293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:26.972293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:26.972293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:26.972293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:26.972293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:26.972293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:26.972293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:26.972293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:26.972293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:26.972293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:26.972293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:26.972293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:26.973200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:26.973200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:26.973200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:26.973200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:26.973200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:26.973200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:26.973200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:26.973200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:26.973200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:26.973200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:26.973200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:26.973200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:26.973200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:26.973200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:26.973200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:26.973200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:26.973200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:26.974037 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:26.974037 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:26.974037 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:26.974037 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:26.982614 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 232/462] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:26.984373 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2042f256-2c00-4ffa-917b-02c921e9e003 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:26.991518 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:27.026561 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:27.081901 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad6707f-28e7-4e71-9e6d-04e22252bce0 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:27.112932 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2042f256-2c00-4ffa-917b-02c921e9e003 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:27.119745 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fe06d1-f105-4df4-96cd-3e168cab6965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:27.120567 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe06d1-f105-4df4-96cd-3e168cab6965 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:27.120966 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe06d1-f105-4df4-96cd-3e168cab6965 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:27.121248 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe06d1-f105-4df4-96cd-3e168cab6965 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:27.132399 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fe06d1-f105-4df4-96cd-3e168cab6965 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:27.133798 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fe06d1-f105-4df4-96cd-3e168cab6965 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:27.154209 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-47fe06d1-f105-4df4-96cd-3e168cab6965 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:27.154209 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:27.154209 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:27.154209 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:27.154209 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:27.154209 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:27.154209 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:27.154209 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:27.154209 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:27.154209 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:27.154209 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:27.154209 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:27.154209 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:27.154209 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:27.154209 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:27.155109 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:27.155109 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:27.155109 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:27.155109 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:27.155109 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:27.155109 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:27.155109 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:27.155109 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:27.155109 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:27.155109 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:27.155109 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:27.155109 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:27.155109 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:27.155109 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:27.155109 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:27.155109 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:27.155109 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:27.156371 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:27.156371 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:27.156371 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:27.156371 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:27.156371 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 231/463] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:27.156371 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fe06d1-f105-4df4-96cd-3e168cab6965 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:27.156893 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a955a3-8175-446b-af5a-ef4b0656759d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:27.157817 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a955a3-8175-446b-af5a-ef4b0656759d None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:27.158245 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a955a3-8175-446b-af5a-ef4b0656759d None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:27.158674 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a955a3-8175-446b-af5a-ef4b0656759d None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:27.159670 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a955a3-8175-446b-af5a-ef4b0656759d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:27.160200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a955a3-8175-446b-af5a-ef4b0656759d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:27.161858 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a955a3-8175-446b-af5a-ef4b0656759d None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:27.181224 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 233/464] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:27.182715 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a955a3-8175-446b-af5a-ef4b0656759d None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:27.258822 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:27.267948 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fe06d1-f105-4df4-96cd-3e168cab6965 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:27.271628 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:27.307236 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b00a68-b218-40c0-bb66-8bbe04ed73da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:27.308612 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b00a68-b218-40c0-bb66-8bbe04ed73da None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:27.308892 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b00a68-b218-40c0-bb66-8bbe04ed73da None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:27.309172 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b00a68-b218-40c0-bb66-8bbe04ed73da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:27.310229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a955a3-8175-446b-af5a-ef4b0656759d None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:27.322535 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b00a68-b218-40c0-bb66-8bbe04ed73da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:27.324750 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b00a68-b218-40c0-bb66-8bbe04ed73da None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:27.349457 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff516d8-8dd3-4248-a49e-73235fa9a924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:27.350178 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-03b00a68-b218-40c0-bb66-8bbe04ed73da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:27.350178 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:27.350178 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:27.350178 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:27.350178 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:27.350178 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:27.350178 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:27.350178 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:27.350178 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:27.350178 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:27.350178 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:27.350178 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:27.350178 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:27.350178 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:27.350178 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:27.351045 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:27.351045 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:27.351045 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:27.351045 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:27.351045 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:27.351045 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:27.351045 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:27.351045 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:27.351045 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:27.351045 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:27.351045 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:27.351045 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:27.351045 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:27.351045 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:27.351045 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:27.351045 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:27.351045 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:27.352008 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:27.352008 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:27.352008 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:27.352008 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:27.353247 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff516d8-8dd3-4248-a49e-73235fa9a924 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:27.353735 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff516d8-8dd3-4248-a49e-73235fa9a924 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:27.354711 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff516d8-8dd3-4248-a49e-73235fa9a924 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:27.356160 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff516d8-8dd3-4248-a49e-73235fa9a924 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:27.356933 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff516d8-8dd3-4248-a49e-73235fa9a924 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:27.357230 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 232/465] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:27.358143 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b00a68-b218-40c0-bb66-8bbe04ed73da None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:27.358986 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff516d8-8dd3-4248-a49e-73235fa9a924 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:27.373610 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 234/466] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:27.374869 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff516d8-8dd3-4248-a49e-73235fa9a924 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:27.471568 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b00a68-b218-40c0-bb66-8bbe04ed73da None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:27.478438 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff516d8-8dd3-4248-a49e-73235fa9a924 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:27.506157 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:27.512306 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cea25e-ca97-496a-a393-d9c900eee513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:27.512952 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cea25e-ca97-496a-a393-d9c900eee513 None admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:27.513216 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cea25e-ca97-496a-a393-d9c900eee513 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:27.513493 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cea25e-ca97-496a-a393-d9c900eee513 None admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:27.514216 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cea25e-ca97-496a-a393-d9c900eee513 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:27.515720 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cea25e-ca97-496a-a393-d9c900eee513 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:27.518968 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500f37ae-3ab8-4b2a-bee5-3a72e3e4360c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:27.519672 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f37ae-3ab8-4b2a-bee5-3a72e3e4360c None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:27.519959 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f37ae-3ab8-4b2a-bee5-3a72e3e4360c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:27.520241 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f37ae-3ab8-4b2a-bee5-3a72e3e4360c None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:27.520965 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500f37ae-3ab8-4b2a-bee5-3a72e3e4360c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:27.521274 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500f37ae-3ab8-4b2a-bee5-3a72e3e4360c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:27.522846 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500f37ae-3ab8-4b2a-bee5-3a72e3e4360c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:27.535656 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 235/467] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:27.536966 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500f37ae-3ab8-4b2a-bee5-3a72e3e4360c None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:27.640411 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500f37ae-3ab8-4b2a-bee5-3a72e3e4360c None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:27.646952 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 233/468] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 18 16:24:27 2022] POST /identity/v3/users => generated 246 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:24:27.657155 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce63448-382e-454b-8a78-9f407b614c64 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:27.657459 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce63448-382e-454b-8a78-9f407b614c64 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:27.657743 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce63448-382e-454b-8a78-9f407b614c64 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:27.659526 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 234/469] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:27.660082 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce63448-382e-454b-8a78-9f407b614c64 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:27.689281 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a79146-9d7b-4b73-8263-129c957fdb41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:27.689513 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:27.690862 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a79146-9d7b-4b73-8263-129c957fdb41 None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:27.691620 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a79146-9d7b-4b73-8263-129c957fdb41 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:27.692186 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a79146-9d7b-4b73-8263-129c957fdb41 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:27.693274 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a79146-9d7b-4b73-8263-129c957fdb41 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:27.695866 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a79146-9d7b-4b73-8263-129c957fdb41 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:27.798659 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 236/470] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 18 16:24:27 2022] POST /identity/v3/users => generated 248 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:24:27.800884 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067b16a-165c-43c7-8241-b36ae37e4343 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:27.801020 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce63448-382e-454b-8a78-9f407b614c64 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:27.801547 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067b16a-165c-43c7-8241-b36ae37e4343 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:27.802091 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067b16a-165c-43c7-8241-b36ae37e4343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:27.836759 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-f067b16a-165c-43c7-8241-b36ae37e4343 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:27.838420 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f067b16a-165c-43c7-8241-b36ae37e4343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:27.866179 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b10360-e729-4bce-beb8-99e95cfcc9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:27.866878 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b10360-e729-4bce-beb8-99e95cfcc9d2 None admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:27.867327 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b10360-e729-4bce-beb8-99e95cfcc9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:27.867598 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b10360-e729-4bce-beb8-99e95cfcc9d2 None admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:27.868388 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b10360-e729-4bce-beb8-99e95cfcc9d2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:27.870787 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b10360-e729-4bce-beb8-99e95cfcc9d2 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:27.887523 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:27.932034 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 237/471] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:27.958114 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f067b16a-165c-43c7-8241-b36ae37e4343 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:28.011662 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 235/472] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 18 16:24:27 2022] POST /identity/v3/users => generated 248 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:24:28.077983 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f067b16a-165c-43c7-8241-b36ae37e4343 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:28.121129 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f73668e-f253-4184-8bc9-e1f9b9058005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:28.122063 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f73668e-f253-4184-8bc9-e1f9b9058005 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:28.122346 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f73668e-f253-4184-8bc9-e1f9b9058005 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:28.122673 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f73668e-f253-4184-8bc9-e1f9b9058005 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:28.136946 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f73668e-f253-4184-8bc9-e1f9b9058005 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:28.138471 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f73668e-f253-4184-8bc9-e1f9b9058005 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:28.158220 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-7f73668e-f253-4184-8bc9-e1f9b9058005 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:28.158220 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:28.158220 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:28.158220 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:28.158220 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:28.158220 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:28.158220 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:28.158220 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:28.158220 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:28.158220 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:28.158220 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:28.158220 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:28.158220 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:28.158220 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:28.158220 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:28.159129 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:28.159129 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:28.159129 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:28.159129 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:28.159129 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:28.159129 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:28.159129 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:28.159129 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:28.159129 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:28.159129 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:28.159129 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:28.159129 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:28.159129 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:28.159129 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:28.159129 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:28.159129 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:28.159129 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:28.160009 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:28.160009 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:28.160009 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:28.160009 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:28.160009 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 238/473] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:24:28.168994 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b10360-e729-4bce-beb8-99e95cfcc9d2 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:28.255164 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96311eae-67f9-4cb5-b395-0a33e7217ee3 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:28.255946 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96311eae-67f9-4cb5-b395-0a33e7217ee3 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:28.256418 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:28.258531 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96311eae-67f9-4cb5-b395-0a33e7217ee3 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:28.259389 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 239/474] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:28.265232 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c016e0-329e-4a97-965d-0265f70a0ce7 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:28.265930 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c016e0-329e-4a97-965d-0265f70a0ce7 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:28.266402 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c016e0-329e-4a97-965d-0265f70a0ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:28.286990 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-b5c016e0-329e-4a97-965d-0265f70a0ce7 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:28.288281 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-b5c016e0-329e-4a97-965d-0265f70a0ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:28.302602 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b10360-e729-4bce-beb8-99e95cfcc9d2 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:28.305103 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:28.343104 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f972be-5f49-41af-87b1-f5e7ee0c35ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:28.343888 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f972be-5f49-41af-87b1-f5e7ee0c35ec None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:28.344180 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f972be-5f49-41af-87b1-f5e7ee0c35ec None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:28.344472 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f972be-5f49-41af-87b1-f5e7ee0c35ec None admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:28.345213 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f972be-5f49-41af-87b1-f5e7ee0c35ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:28.345528 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f972be-5f49-41af-87b1-f5e7ee0c35ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:28.346934 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f972be-5f49-41af-87b1-f5e7ee0c35ec None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:28.366916 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 236/475] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:28.372725 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f972be-5f49-41af-87b1-f5e7ee0c35ec None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:28.377833 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 240/476] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:28.405227 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c016e0-329e-4a97-965d-0265f70a0ce7 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:28.484974 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f972be-5f49-41af-87b1-f5e7ee0c35ec None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:28.514913 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:28.524728 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c52213-cca3-468d-a6d6-1b24760f4b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:28.527297 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c52213-cca3-468d-a6d6-1b24760f4b87 None admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:28.527297 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c52213-cca3-468d-a6d6-1b24760f4b87 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:28.527297 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c52213-cca3-468d-a6d6-1b24760f4b87 None admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:28.527629 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c52213-cca3-468d-a6d6-1b24760f4b87 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:28.529284 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c52213-cca3-468d-a6d6-1b24760f4b87 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:28.601212 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c016e0-329e-4a97-965d-0265f70a0ce7 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:28.620541 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 237/477] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 18 16:24:28 2022] POST /identity/v3/users => generated 249 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:24:28.652544 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de94e72-5245-4891-a6d9-e20d56840878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:28.653713 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de94e72-5245-4891-a6d9-e20d56840878 None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:28.654166 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de94e72-5245-4891-a6d9-e20d56840878 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:28.654626 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de94e72-5245-4891-a6d9-e20d56840878 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:28.655698 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de94e72-5245-4891-a6d9-e20d56840878 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:28.657427 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de94e72-5245-4891-a6d9-e20d56840878 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:28.703123 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.sql.core [None req-7de94e72-5245-4891-a6d9-e20d56840878 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 18 16:24:28.703123 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 18 16:24:28.703123 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [parameters: {'id': '700d9d7ada09435284f3ed091ba729e4', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 18 16:24:28.703123 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80368) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 18 16:24:28.726429 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-7de94e72-5245-4891-a6d9-e20d56840878 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 18 16:24:28.726429 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:28.726429 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 16:24:28.726429 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 16:24:28.726429 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 16:24:28.726429 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 16:24:28.726429 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 16:24:28.726429 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 16:24:28.726429 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 16:24:28.726429 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application conn.query(q) Jul 18 16:24:28.726429 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 16:24:28.726429 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 16:24:28.726429 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 16:24:28.726429 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application result.read() Jul 18 16:24:28.726429 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 16:24:28.727575 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 16:24:28.727575 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 16:24:28.727575 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 16:24:28.727575 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 16:24:28.727575 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 16:24:28.727575 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 16:24:28.727575 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 16:24:28.727575 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 18 16:24:28.727575 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:28.727575 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 18 16:24:28.727575 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:28.727575 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:28.727575 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 18 16:24:28.727575 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 18 16:24:28.727575 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 18 16:24:28.727575 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return ref.to_dict() Jul 18 16:24:28.727575 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 16:24:28.727575 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application next(self.gen) Jul 18 16:24:28.728730 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 18 16:24:28.728730 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application yield resource Jul 18 16:24:28.728730 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 16:24:28.728730 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application next(self.gen) Jul 18 16:24:28.728730 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 18 16:24:28.728730 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self.session.rollback() Jul 18 16:24:28.728730 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 18 16:24:28.728730 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self.force_reraise() Jul 18 16:24:28.728730 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 18 16:24:28.728730 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise self.value Jul 18 16:24:28.728730 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 18 16:24:28.728730 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 18 16:24:28.728730 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 18 16:24:28.728730 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application session.commit() Jul 18 16:24:28.728730 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 18 16:24:28.728730 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 18 16:24:28.728730 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 18 16:24:28.730460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._prepare_impl() Jul 18 16:24:28.730460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 18 16:24:28.730460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self.session.flush() Jul 18 16:24:28.730460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 18 16:24:28.730460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._flush(objects) Jul 18 16:24:28.730460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 18 16:24:28.730460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 18 16:24:28.730460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 18 16:24:28.730460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application compat.raise_( Jul 18 16:24:28.730460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 16:24:28.730460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception Jul 18 16:24:28.730460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 18 16:24:28.730460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application flush_context.execute() Jul 18 16:24:28.730460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 18 16:24:28.730460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rec.execute(self) Jul 18 16:24:28.730460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 18 16:24:28.730460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 18 16:24:28.731705 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 18 16:24:28.731705 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 18 16:24:28.731705 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 18 16:24:28.731705 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 18 16:24:28.731705 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 18 16:24:28.731705 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 18 16:24:28.731705 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 18 16:24:28.731705 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 18 16:24:28.731705 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 18 16:24:28.731705 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 18 16:24:28.731705 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 18 16:24:28.731705 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 18 16:24:28.731705 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 18 16:24:28.731705 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 18 16:24:28.731705 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 16:24:28.731705 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception Jul 18 16:24:28.732692 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 16:24:28.732692 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 16:24:28.732692 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 16:24:28.732692 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 16:24:28.732692 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 16:24:28.732692 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 16:24:28.732692 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 16:24:28.732692 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application conn.query(q) Jul 18 16:24:28.732692 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 16:24:28.732692 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 16:24:28.732692 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 16:24:28.732692 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application result.read() Jul 18 16:24:28.732692 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 16:24:28.732692 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 16:24:28.732692 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 16:24:28.732692 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 16:24:28.733875 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 16:24:28.733875 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 16:24:28.733875 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 16:24:28.733875 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 16:24:28.733875 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 18 16:24:28.733875 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 18 16:24:28.733875 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [parameters: {'id': '700d9d7ada09435284f3ed091ba729e4', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 18 16:24:28.733875 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 18 16:24:28.733875 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:28.733875 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:24:28.733875 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:28.733875 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:28.733875 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:28.733875 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:28.733875 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:28.733875 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:28.734768 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:28.734768 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:28.734768 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:28.734768 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:28.734768 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:28.734768 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:28.734768 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 18 16:24:28.734768 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 18 16:24:28.734768 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:28.734768 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:28.734768 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 18 16:24:28.734768 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 18 16:24:28.734768 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 18 16:24:28.734768 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 18 16:24:28.734768 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 18 16:24:28.734768 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:28.737522 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 241/478] 10.222.0.26 () {60 vars in 1131 bytes} [Mon Jul 18 16:24:28 2022] POST /identity/v3/roles => generated 144 bytes in 333 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:24:28.745417 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c52213-cca3-468d-a6d6-1b24760f4b87 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:28.869911 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c52213-cca3-468d-a6d6-1b24760f4b87 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:28.907972 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b097990-bbae-4846-96aa-8c814be32b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:28.909117 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b097990-bbae-4846-96aa-8c814be32b8d None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:28.909582 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b097990-bbae-4846-96aa-8c814be32b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:28.910041 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b097990-bbae-4846-96aa-8c814be32b8d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:28.922238 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b097990-bbae-4846-96aa-8c814be32b8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:28.930646 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-5b097990-bbae-4846-96aa-8c814be32b8d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:24:28.930646 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:28.930646 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:28.930646 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:28.930646 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:28.930646 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:28.930646 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:28.930646 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:28.930646 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:28.930646 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:28.930646 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:28.930646 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:28.930646 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:28.930646 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:28.930646 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:28.930646 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:28.931568 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:28.931568 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:28.931568 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:28.931568 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:28.931568 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:28.931568 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:28.931568 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:28.931568 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:28.931568 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:28.931568 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:28.931568 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:28.931568 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:28.931568 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:28.931568 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:28.931568 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:28.931568 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:28.932930 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:28.932930 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:28.932930 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:28.932930 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:28.932930 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:28.932930 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:28.932930 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:28.932930 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:28.932930 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:28.932930 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:28.932930 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:24:28.932930 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:28.935374 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 238/479] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 18 16:24:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:28.940750 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de94e72-5245-4891-a6d9-e20d56840878 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:28.960054 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:29.060061 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de94e72-5245-4891-a6d9-e20d56840878 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:29.097329 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9264a7-f122-4b4f-a08e-590edeaed574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:29.098009 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9264a7-f122-4b4f-a08e-590edeaed574 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:29.098294 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9264a7-f122-4b4f-a08e-590edeaed574 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:29.098569 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9264a7-f122-4b4f-a08e-590edeaed574 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:29.099321 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9264a7-f122-4b4f-a08e-590edeaed574 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:29.099660 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9264a7-f122-4b4f-a08e-590edeaed574 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:29.101041 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9264a7-f122-4b4f-a08e-590edeaed574 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:29.112322 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 242/480] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:24:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:30.306860 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff16c1f0-263a-4421-8762-8704c62f8834 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:30.308472 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff16c1f0-263a-4421-8762-8704c62f8834 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:30.308982 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff16c1f0-263a-4421-8762-8704c62f8834 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:30.310965 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 239/481] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:30.316710 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106508c4-ec11-4e3d-8ff1-eeedc4794cbf None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:30.317833 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106508c4-ec11-4e3d-8ff1-eeedc4794cbf None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:30.318335 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106508c4-ec11-4e3d-8ff1-eeedc4794cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:30.340210 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-106508c4-ec11-4e3d-8ff1-eeedc4794cbf None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:30.341764 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-106508c4-ec11-4e3d-8ff1-eeedc4794cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:30.416479 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686378c8-a02e-42df-abfd-ca1c41435072 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:30.417420 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686378c8-a02e-42df-abfd-ca1c41435072 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:30.418032 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686378c8-a02e-42df-abfd-ca1c41435072 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:30.421256 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 240/482] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:30.426585 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c6253-d67f-4228-a43e-b27f9d197a6c None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:30.426988 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c6253-d67f-4228-a43e-b27f9d197a6c None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:30.427396 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c6253-d67f-4228-a43e-b27f9d197a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:30.442039 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 243/483] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:30.460903 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-d49c6253-d67f-4228-a43e-b27f9d197a6c None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:30.462766 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-d49c6253-d67f-4228-a43e-b27f9d197a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:30.468688 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106508c4-ec11-4e3d-8ff1-eeedc4794cbf None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:30.484077 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:30.517148 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:30.549129 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 241/484] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:30.574762 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c6253-d67f-4228-a43e-b27f9d197a6c None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:30.695872 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c6253-d67f-4228-a43e-b27f9d197a6c None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:30.701042 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106508c4-ec11-4e3d-8ff1-eeedc4794cbf None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:30.740119 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7dd22b-ce85-4e91-9e2c-12b859f62068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:30.740917 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7dd22b-ce85-4e91-9e2c-12b859f62068 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:30.741201 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7dd22b-ce85-4e91-9e2c-12b859f62068 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:30.741481 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7dd22b-ce85-4e91-9e2c-12b859f62068 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:30.743931 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371c56b9-f027-4444-871f-2e209d71c8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:30.744811 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c56b9-f027-4444-871f-2e209d71c8d4 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:30.745225 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c56b9-f027-4444-871f-2e209d71c8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:30.745677 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c56b9-f027-4444-871f-2e209d71c8d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:30.754367 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7dd22b-ce85-4e91-9e2c-12b859f62068 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:30.756635 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371c56b9-f027-4444-871f-2e209d71c8d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:30.762861 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-3a7dd22b-ce85-4e91-9e2c-12b859f62068 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:30.762861 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:30.762861 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:30.762861 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:30.762861 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:30.762861 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:30.762861 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:30.762861 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:30.762861 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:30.762861 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:30.762861 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:30.762861 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:30.762861 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:30.762861 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:30.762861 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:30.762861 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:30.763813 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:30.763813 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:30.763813 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:30.763813 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:30.763813 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:30.763813 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:30.763813 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:30.763813 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:30.763813 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:30.763813 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:30.763813 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:30.763813 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:30.763813 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:30.763813 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:30.763813 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:30.763813 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:30.765013 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:30.765013 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:30.765013 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:30.765013 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:30.765013 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:30.765013 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:30.765013 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:30.765013 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:30.765013 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:30.765013 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:30.765013 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:30.765013 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:30.765897 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-371c56b9-f027-4444-871f-2e209d71c8d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:30.765897 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:30.765897 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:30.765897 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:30.765897 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:30.765897 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:30.765897 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:30.765897 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:30.765897 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:30.765897 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:30.765897 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:30.765897 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:30.765897 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:30.765897 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:30.765897 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:30.765897 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:30.766696 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:30.766696 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:30.766696 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:30.766696 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:30.766696 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:30.766696 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:30.766696 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:30.766696 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:30.766696 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:30.766696 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:30.766696 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:30.766696 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:30.766696 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:30.766696 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:30.766696 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:30.766696 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:30.767505 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:30.767505 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:30.767505 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:30.767505 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:30.767505 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:30.767505 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:30.767505 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:30.767505 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:30.767505 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:30.767505 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:30.767505 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:30.767505 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:30.769310 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 242/486] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 18 16:24:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:24:30.769310 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69866f92-dcda-4b67-bfe8-55d5c54170bd None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:30.769601 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69866f92-dcda-4b67-bfe8-55d5c54170bd None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:30.772574 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 244/486] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 18 16:24:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:24:30.772698 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69866f92-dcda-4b67-bfe8-55d5c54170bd None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:30.772804 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7dd22b-ce85-4e91-9e2c-12b859f62068 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:30.772891 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 243/487] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:30.778091 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a14e31-f585-4fdc-9f37-7fe63749a8b6 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:30.780045 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a14e31-f585-4fdc-9f37-7fe63749a8b6 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:30.780045 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a14e31-f585-4fdc-9f37-7fe63749a8b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:30.797679 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-c5a14e31-f585-4fdc-9f37-7fe63749a8b6 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:30.798868 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-c5a14e31-f585-4fdc-9f37-7fe63749a8b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:30.843779 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:30.871983 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:30.908216 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 244/488] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:24:30.909431 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a14e31-f585-4fdc-9f37-7fe63749a8b6 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:30.917712 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7dd22b-ce85-4e91-9e2c-12b859f62068 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:30.960954 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eecc118-ccac-4b77-a07d-929d87e13498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:30.962071 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eecc118-ccac-4b77-a07d-929d87e13498 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:30.962586 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eecc118-ccac-4b77-a07d-929d87e13498 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:30.963059 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eecc118-ccac-4b77-a07d-929d87e13498 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:30.965028 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eecc118-ccac-4b77-a07d-929d87e13498 None admin] RBAC: Adding query filter params (name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:30.965554 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eecc118-ccac-4b77-a07d-929d87e13498 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:30.967272 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eecc118-ccac-4b77-a07d-929d87e13498 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:30.976833 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 245/489] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 18 16:24:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:30.978086 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eecc118-ccac-4b77-a07d-929d87e13498 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:31.037733 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a14e31-f585-4fdc-9f37-7fe63749a8b6 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:31.078047 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9569524-59f8-457b-93d7-1022e72f6035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:31.078795 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9569524-59f8-457b-93d7-1022e72f6035 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:31.079014 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9569524-59f8-457b-93d7-1022e72f6035 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:31.079298 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9569524-59f8-457b-93d7-1022e72f6035 None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:31.080082 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9569524-59f8-457b-93d7-1022e72f6035 None admin] RBAC: Adding query filter params (name=service) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:31.080398 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9569524-59f8-457b-93d7-1022e72f6035 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:31.082000 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9569524-59f8-457b-93d7-1022e72f6035 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:31.091804 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 245/490] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 18 16:24:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:31.092518 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eecc118-ccac-4b77-a07d-929d87e13498 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:31.092628 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9569524-59f8-457b-93d7-1022e72f6035 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:31.119711 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:31.122928 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:31.131452 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8b5eaa-0293-4e66-a076-1c5b0539829f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:31.139023 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8b5eaa-0293-4e66-a076-1c5b0539829f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:31.139023 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8b5eaa-0293-4e66-a076-1c5b0539829f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:31.139023 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8b5eaa-0293-4e66-a076-1c5b0539829f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:31.147200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8b5eaa-0293-4e66-a076-1c5b0539829f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:31.156189 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-1f8b5eaa-0293-4e66-a076-1c5b0539829f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:31.156189 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:31.156189 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:31.156189 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:31.156189 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:31.156189 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:31.156189 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:31.156189 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:31.156189 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:31.156189 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:31.156189 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:31.156189 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:31.156189 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:31.156189 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:31.156189 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:31.156189 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:31.157134 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:31.157134 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:31.157134 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:31.157134 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:31.157134 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:31.157134 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:31.157134 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:31.157134 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:31.157134 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:31.157134 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:31.157134 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:31.157134 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:31.157134 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:31.157134 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:31.157134 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:31.157134 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:31.157939 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:31.157939 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:31.157939 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:31.157939 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:31.157939 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:31.157939 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:31.157939 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:31.157939 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:31.157939 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:31.157939 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:31.157939 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:31.157939 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:31.162449 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 246/491] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 18 16:24:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:31.163345 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8b5eaa-0293-4e66-a076-1c5b0539829f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:31.214840 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9569524-59f8-457b-93d7-1022e72f6035 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:31.254810 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1ffa06-5d6c-4aec-a721-62dec95dd17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:31.256126 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1ffa06-5d6c-4aec-a721-62dec95dd17d None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:31.256615 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1ffa06-5d6c-4aec-a721-62dec95dd17d None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:31.257185 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1ffa06-5d6c-4aec-a721-62dec95dd17d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:31.269721 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1ffa06-5d6c-4aec-a721-62dec95dd17d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:31.271578 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1ffa06-5d6c-4aec-a721-62dec95dd17d None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:31.282666 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8b5eaa-0293-4e66-a076-1c5b0539829f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:31.292808 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-ad1ffa06-5d6c-4aec-a721-62dec95dd17d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:31.292808 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:31.292808 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:31.292808 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:31.292808 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:31.292808 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:31.292808 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:31.292808 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:31.292808 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:31.292808 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:31.292808 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:31.292808 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:31.292808 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:31.292808 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:31.292808 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:31.293663 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:31.293663 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:31.293663 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:31.293663 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:31.293663 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:31.293663 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:31.293663 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:31.293663 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:31.293663 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:31.293663 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:31.293663 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:31.293663 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:31.293663 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:31.293663 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:31.293663 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:31.293663 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:31.293663 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:31.294784 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:31.294784 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:31.294784 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:31.294784 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:31.297158 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 246/492] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:31.298407 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1ffa06-5d6c-4aec-a721-62dec95dd17d None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:31.321073 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b0c5ac-e15c-410f-9c8d-b8a6ba7701ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:31.321902 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b0c5ac-e15c-410f-9c8d-b8a6ba7701ea None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:31.322335 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b0c5ac-e15c-410f-9c8d-b8a6ba7701ea None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:31.322533 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b0c5ac-e15c-410f-9c8d-b8a6ba7701ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:31.334955 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b0c5ac-e15c-410f-9c8d-b8a6ba7701ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:31.336269 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b0c5ac-e15c-410f-9c8d-b8a6ba7701ea None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:31.357374 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-44b0c5ac-e15c-410f-9c8d-b8a6ba7701ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:31.357374 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:31.357374 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:31.357374 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:31.357374 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:31.357374 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:31.357374 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:31.357374 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:31.357374 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:31.357374 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:31.357374 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:31.357374 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:31.357374 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:31.357374 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:31.357374 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:31.358278 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:31.358278 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:31.358278 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:31.358278 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:31.358278 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:31.358278 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:31.358278 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:31.358278 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:31.358278 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:31.358278 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:31.358278 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:31.358278 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:31.358278 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:31.358278 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:31.358278 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:31.358278 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:31.358278 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:31.359812 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:31.359812 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:31.359812 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:31.359812 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:31.359812 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 247/493] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:31.359812 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b0c5ac-e15c-410f-9c8d-b8a6ba7701ea None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:31.394420 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:31.406397 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:31.421382 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1ffa06-5d6c-4aec-a721-62dec95dd17d None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:31.459095 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3184d9ff-214e-4287-9cd9-2e9b866f0089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:31.460186 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184d9ff-214e-4287-9cd9-2e9b866f0089 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:31.460675 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184d9ff-214e-4287-9cd9-2e9b866f0089 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:31.461213 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184d9ff-214e-4287-9cd9-2e9b866f0089 None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:31.462216 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3184d9ff-214e-4287-9cd9-2e9b866f0089 None admin] RBAC: Adding query filter params (name=service) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:31.462751 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3184d9ff-214e-4287-9cd9-2e9b866f0089 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:31.464412 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3184d9ff-214e-4287-9cd9-2e9b866f0089 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:31.475634 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 247/494] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 18 16:24:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:31.476842 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3184d9ff-214e-4287-9cd9-2e9b866f0089 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:31.519101 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b0c5ac-e15c-410f-9c8d-b8a6ba7701ea None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:31.557018 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8113cd-6694-4f70-a721-678b703ddf13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:31.558681 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8113cd-6694-4f70-a721-678b703ddf13 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:31.559167 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8113cd-6694-4f70-a721-678b703ddf13 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:31.559606 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8113cd-6694-4f70-a721-678b703ddf13 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:31.560416 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8113cd-6694-4f70-a721-678b703ddf13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:31.560825 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8113cd-6694-4f70-a721-678b703ddf13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:31.562319 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8113cd-6694-4f70-a721-678b703ddf13 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:31.576209 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 248/495] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:31.577060 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8113cd-6694-4f70-a721-678b703ddf13 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:31.601245 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3184d9ff-214e-4287-9cd9-2e9b866f0089 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:31.638845 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb84ef77-f769-45d5-aafc-d4fa01427517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:31.639569 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb84ef77-f769-45d5-aafc-d4fa01427517 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:31.639927 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb84ef77-f769-45d5-aafc-d4fa01427517 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:31.640277 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb84ef77-f769-45d5-aafc-d4fa01427517 None admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:31.641079 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb84ef77-f769-45d5-aafc-d4fa01427517 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:31.641446 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb84ef77-f769-45d5-aafc-d4fa01427517 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:31.642901 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb84ef77-f769-45d5-aafc-d4fa01427517 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:31.651820 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:31.661906 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 248/496] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:31.661906 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb5069-5e67-4d0e-9c5b-3cae36c4b4a8 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:31.661906 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb5069-5e67-4d0e-9c5b-3cae36c4b4a8 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:31.661906 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb5069-5e67-4d0e-9c5b-3cae36c4b4a8 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:31.661906 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 249/497] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:31.661906 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeb5069-5e67-4d0e-9c5b-3cae36c4b4a8 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:31.687832 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:31.691031 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8113cd-6694-4f70-a721-678b703ddf13 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:31.727551 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea9c417-bdd7-41b1-b318-cc3e2ba22344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:31.728443 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea9c417-bdd7-41b1-b318-cc3e2ba22344 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:31.728800 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea9c417-bdd7-41b1-b318-cc3e2ba22344 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:31.729148 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea9c417-bdd7-41b1-b318-cc3e2ba22344 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:31.740718 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea9c417-bdd7-41b1-b318-cc3e2ba22344 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:31.742320 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea9c417-bdd7-41b1-b318-cc3e2ba22344 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:31.762276 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-6ea9c417-bdd7-41b1-b318-cc3e2ba22344 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:31.762276 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:31.762276 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:31.762276 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:31.762276 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:31.762276 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:31.762276 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:31.762276 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:31.762276 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:31.762276 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:31.762276 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:31.762276 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:31.762276 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:31.762276 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:31.762276 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:31.763371 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:31.763371 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:31.763371 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:31.763371 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:31.763371 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:31.763371 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:31.763371 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:31.763371 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:31.763371 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:31.763371 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:31.763371 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:31.763371 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:31.763371 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:31.763371 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:31.763371 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:31.763371 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:31.763371 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:31.764803 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:31.764803 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:31.764803 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:31.764803 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:31.764803 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 249/498] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:31.765159 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea9c417-bdd7-41b1-b318-cc3e2ba22344 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:31.776205 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeb5069-5e67-4d0e-9c5b-3cae36c4b4a8 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:31.814307 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed784a3e-9732-447b-9663-031d274799e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:31.815473 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed784a3e-9732-447b-9663-031d274799e8 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:31.815979 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed784a3e-9732-447b-9663-031d274799e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:31.816386 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed784a3e-9732-447b-9663-031d274799e8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:31.824540 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed784a3e-9732-447b-9663-031d274799e8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:31.825987 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed784a3e-9732-447b-9663-031d274799e8 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:31.850205 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-ed784a3e-9732-447b-9663-031d274799e8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:24:31.850205 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:31.850205 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:31.850205 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:31.850205 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:31.850205 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:31.850205 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:31.850205 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:31.850205 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:31.850205 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:31.850205 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:31.850205 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:31.850205 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:24:31.850205 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:24:31.850205 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:24:31.850205 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:24:31.851173 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:31.851173 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:31.851173 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:24:31.851173 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:31.851173 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:24:31.851173 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:31.851173 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:31.851173 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:31.851173 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:31.851173 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:31.851173 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:31.851173 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:31.851173 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:31.851173 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:31.851173 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:31.851173 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:31.851173 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:31.851173 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:31.852912 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:31.852912 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:31.852912 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:24:31.852912 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:24:31.852912 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:24:31.852912 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:24:31.852912 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:24:31.852912 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:24:31.852912 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:24:31.852912 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:31.852912 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 250/499] 10.222.0.26 () {56 vars in 1093 bytes} [Mon Jul 18 16:24:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:31.852912 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ee823-e849-4000-b75f-3bb5e76bcef0 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:31.852912 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ee823-e849-4000-b75f-3bb5e76bcef0 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:31.853577 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ee823-e849-4000-b75f-3bb5e76bcef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:31.872099 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-9e5ee823-e849-4000-b75f-3bb5e76bcef0 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:31.873413 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-9e5ee823-e849-4000-b75f-3bb5e76bcef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:31.876435 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea9c417-bdd7-41b1-b318-cc3e2ba22344 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:31.909675 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90ed62d-984b-4428-8c3f-a727b520a372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:31.910384 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90ed62d-984b-4428-8c3f-a727b520a372 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:31.910662 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90ed62d-984b-4428-8c3f-a727b520a372 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:31.910933 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90ed62d-984b-4428-8c3f-a727b520a372 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:31.918761 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90ed62d-984b-4428-8c3f-a727b520a372 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:31.920515 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90ed62d-984b-4428-8c3f-a727b520a372 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:31.926132 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:31.937049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-c90ed62d-984b-4428-8c3f-a727b520a372 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 18 16:24:31.937049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:31.937049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:31.937049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:31.937049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:31.937049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:31.937049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:31.937049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:31.937049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:31.937049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:31.937049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:31.937049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:31.937049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:24:31.937049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:24:31.937049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:24:31.937049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:24:31.946714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:31.946714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:31.946714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:24:31.946714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:31.946714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:24:31.946714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:31.946714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:31.946714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:31.946714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:31.946714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:31.946714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:31.946714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:31.946714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:31.946714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:31.946714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:31.946714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:31.946714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:31.946714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:31.947652 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:31.947652 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:31.947652 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:24:31.947652 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:24:31.947652 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:24:31.947652 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:24:31.947652 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:24:31.947652 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:24:31.947652 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 18 16:24:31.947652 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:31.947652 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 250/500] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:24:31 2022] GET /identity/v3/users/glance => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:31.947652 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90ed62d-984b-4428-8c3f-a727b520a372 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:31.948295 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 251/501] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:31.948295 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77160d3b-778c-4a2e-ba53-2caa928813ff None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:31.948295 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77160d3b-778c-4a2e-ba53-2caa928813ff None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:31.948295 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77160d3b-778c-4a2e-ba53-2caa928813ff None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:31.948295 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 252/502] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:31.948295 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77160d3b-778c-4a2e-ba53-2caa928813ff None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:31.957944 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:32.046268 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90ed62d-984b-4428-8c3f-a727b520a372 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:32.058480 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77160d3b-778c-4a2e-ba53-2caa928813ff None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:32.084924 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9776ac-e45f-44f5-8254-78e26488bed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:32.085597 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9776ac-e45f-44f5-8254-78e26488bed4 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:32.085878 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9776ac-e45f-44f5-8254-78e26488bed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:32.086152 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9776ac-e45f-44f5-8254-78e26488bed4 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:32.086875 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9776ac-e45f-44f5-8254-78e26488bed4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:32.087182 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9776ac-e45f-44f5-8254-78e26488bed4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:32.088626 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9776ac-e45f-44f5-8254-78e26488bed4 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:32.100171 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 251/503] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:32.100745 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a7150-ab59-48f4-9af7-02d5004c835c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:32.101008 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9776ac-e45f-44f5-8254-78e26488bed4 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:32.101506 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a7150-ab59-48f4-9af7-02d5004c835c None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:32.101799 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a7150-ab59-48f4-9af7-02d5004c835c None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:32.102081 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a7150-ab59-48f4-9af7-02d5004c835c None admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:32.102940 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a7150-ab59-48f4-9af7-02d5004c835c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:32.103256 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a7150-ab59-48f4-9af7-02d5004c835c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:32.108712 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a7150-ab59-48f4-9af7-02d5004c835c None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:32.143540 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 253/504] 10.222.0.26 () {56 vars in 1133 bytes} [Mon Jul 18 16:24:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:32.144937 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab3f9f-6c91-4f33-933e-b83dcc90f485 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:32.145235 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab3f9f-6c91-4f33-933e-b83dcc90f485 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:32.145509 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab3f9f-6c91-4f33-933e-b83dcc90f485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:32.162066 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-e2ab3f9f-6c91-4f33-933e-b83dcc90f485 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:32.163197 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-e2ab3f9f-6c91-4f33-933e-b83dcc90f485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:32.177102 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:32.208080 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9776ac-e45f-44f5-8254-78e26488bed4 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:32.209532 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:32.234512 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 254/505] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:32.235355 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ab3f9f-6c91-4f33-933e-b83dcc90f485 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:32.263353 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad983142-a7cf-47f5-80b9-5780f8bf7a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:32.264065 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad983142-a7cf-47f5-80b9-5780f8bf7a42 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:32.264351 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad983142-a7cf-47f5-80b9-5780f8bf7a42 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:32.264629 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad983142-a7cf-47f5-80b9-5780f8bf7a42 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:32.265466 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad983142-a7cf-47f5-80b9-5780f8bf7a42 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:32.265781 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad983142-a7cf-47f5-80b9-5780f8bf7a42 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:32.267221 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad983142-a7cf-47f5-80b9-5780f8bf7a42 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:32.303558 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 252/506] 10.222.0.26 () {56 vars in 1137 bytes} [Mon Jul 18 16:24:32 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:32.304399 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad983142-a7cf-47f5-80b9-5780f8bf7a42 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:32.351258 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ab3f9f-6c91-4f33-933e-b83dcc90f485 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:32.388049 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8622e85b-ed05-4a4b-9e79-abeffb2ec9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:32.388858 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622e85b-ed05-4a4b-9e79-abeffb2ec9f1 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:32.389154 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622e85b-ed05-4a4b-9e79-abeffb2ec9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:32.389434 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622e85b-ed05-4a4b-9e79-abeffb2ec9f1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:32.400031 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8622e85b-ed05-4a4b-9e79-abeffb2ec9f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:32.404741 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad983142-a7cf-47f5-80b9-5780f8bf7a42 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:32.407900 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-8622e85b-ed05-4a4b-9e79-abeffb2ec9f1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:32.407900 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:32.407900 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:32.407900 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:32.407900 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:32.407900 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:32.407900 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:32.407900 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:32.407900 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:32.407900 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:32.407900 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:32.407900 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:32.407900 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:32.407900 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:32.407900 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:32.407900 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:32.408853 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:32.408853 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:32.408853 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:32.408853 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:32.408853 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:32.408853 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:32.408853 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:32.408853 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:32.408853 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:32.408853 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:32.408853 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:32.408853 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:32.408853 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:32.408853 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:32.408853 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:32.408853 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:32.409652 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:32.409652 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:32.409652 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:32.409652 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:32.409652 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:32.409652 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:32.409652 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:32.409652 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:32.409652 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:32.409652 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:32.409652 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:32.409652 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:32.409652 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 255/507] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 18 16:24:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:32.410906 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8622e85b-ed05-4a4b-9e79-abeffb2ec9f1 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:32.438458 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:32.445430 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3a8874-d858-4fb0-b05e-b59c597ede36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:32.446427 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a8874-d858-4fb0-b05e-b59c597ede36 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:32.446751 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a8874-d858-4fb0-b05e-b59c597ede36 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:32.447557 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a8874-d858-4fb0-b05e-b59c597ede36 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:32.450933 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:32.456028 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3a8874-d858-4fb0-b05e-b59c597ede36 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:32.457609 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3a8874-d858-4fb0-b05e-b59c597ede36 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:32.473340 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-be3a8874-d858-4fb0-b05e-b59c597ede36 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 16:24:32.473340 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:32.473340 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:32.473340 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:32.473340 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:32.473340 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:32.473340 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:32.473340 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:32.473340 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:32.473340 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:32.473340 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:32.473340 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:32.473340 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:24:32.473340 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:24:32.473340 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:24:32.473340 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:24:32.477175 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:32.477175 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:32.477175 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:24:32.477175 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:32.477175 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:24:32.477175 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:32.477175 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:32.477175 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:32.477175 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:32.477175 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:32.477175 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:32.477175 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:32.477175 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:32.477175 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:32.477175 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:32.477175 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:32.477175 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:32.477175 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:32.478142 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:32.478142 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:32.478142 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:24:32.478142 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:24:32.478142 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:24:32.478142 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:24:32.478142 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:24:32.478142 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:24:32.478142 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 16:24:32.478142 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:32.478142 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 253/508] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:24:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:32.478142 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3a8874-d858-4fb0-b05e-b59c597ede36 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:32.522875 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8622e85b-ed05-4a4b-9e79-abeffb2ec9f1 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:32.557210 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db50f9da-b6b5-4d98-a7e7-ea7c35d205da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:32.558682 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50f9da-b6b5-4d98-a7e7-ea7c35d205da None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:32.558964 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50f9da-b6b5-4d98-a7e7-ea7c35d205da None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:32.559246 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50f9da-b6b5-4d98-a7e7-ea7c35d205da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:32.579907 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3a8874-d858-4fb0-b05e-b59c597ede36 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:32.622667 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b6be5d-4266-4898-89ee-72ee991a3355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:32.623481 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b6be5d-4266-4898-89ee-72ee991a3355 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:32.623789 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b6be5d-4266-4898-89ee-72ee991a3355 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:32.624063 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b6be5d-4266-4898-89ee-72ee991a3355 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:32.630842 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db50f9da-b6b5-4d98-a7e7-ea7c35d205da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:32.632349 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db50f9da-b6b5-4d98-a7e7-ea7c35d205da None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:32.634051 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b6be5d-4266-4898-89ee-72ee991a3355 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:32.635502 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b6be5d-4266-4898-89ee-72ee991a3355 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:32.657295 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-a1b6be5d-4266-4898-89ee-72ee991a3355 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:32.657295 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:32.657295 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:32.657295 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:32.657295 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:32.657295 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:32.657295 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:32.657295 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:32.657295 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:32.657295 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:32.657295 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:32.657295 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:32.657295 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:32.657295 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:32.657295 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:32.658347 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:32.658347 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:32.658347 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:32.658347 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:32.658347 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:32.658347 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:32.658347 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:32.658347 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:32.658347 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:32.658347 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:32.658347 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:32.658347 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:32.658347 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:32.658347 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:32.658347 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:32.658347 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:32.658347 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:32.659222 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:32.659222 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:32.659222 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:32.659222 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:32.659222 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 254/509] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:32.659486 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b6be5d-4266-4898-89ee-72ee991a3355 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:32.670368 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:32.703186 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:32.736290 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 256/510] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:24:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:24:32.736950 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db50f9da-b6b5-4d98-a7e7-ea7c35d205da None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:32.763949 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b6be5d-4266-4898-89ee-72ee991a3355 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:32.798132 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f7151b-5977-42a2-8b8c-5499814d713b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:32.798780 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7151b-5977-42a2-8b8c-5499814d713b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:32.799059 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7151b-5977-42a2-8b8c-5499814d713b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:32.799341 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7151b-5977-42a2-8b8c-5499814d713b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:32.841112 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db50f9da-b6b5-4d98-a7e7-ea7c35d205da None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:32.872702 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f7151b-5977-42a2-8b8c-5499814d713b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:32.874161 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f7151b-5977-42a2-8b8c-5499814d713b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:32.874977 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ecbf6-f421-4b38-a865-421d208170fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:32.875687 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ecbf6-f421-4b38-a865-421d208170fc None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:32.875979 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ecbf6-f421-4b38-a865-421d208170fc None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:32.876256 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ecbf6-f421-4b38-a865-421d208170fc None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:32.877005 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064ecbf6-f421-4b38-a865-421d208170fc None admin] RBAC: Adding query filter params (name=service) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:32.877312 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064ecbf6-f421-4b38-a865-421d208170fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:32.878690 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064ecbf6-f421-4b38-a865-421d208170fc None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:32.880973 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:32.888605 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 257/511] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 18 16:24:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:32.889404 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ecbf6-f421-4b38-a865-421d208170fc None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:32.917031 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:32.979632 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 255/512] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:24:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:24:32.980295 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f7151b-5977-42a2-8b8c-5499814d713b None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:32.994359 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ecbf6-f421-4b38-a865-421d208170fc None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:33.033563 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df11a953-ebe2-49df-a2cf-3c2411dadc6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:33.034251 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df11a953-ebe2-49df-a2cf-3c2411dadc6d None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:33.034536 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df11a953-ebe2-49df-a2cf-3c2411dadc6d None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:33.034809 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df11a953-ebe2-49df-a2cf-3c2411dadc6d None admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:33.035695 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df11a953-ebe2-49df-a2cf-3c2411dadc6d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:33.036004 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df11a953-ebe2-49df-a2cf-3c2411dadc6d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:33.037463 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df11a953-ebe2-49df-a2cf-3c2411dadc6d None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:33.057263 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 258/513] 10.222.0.26 () {56 vars in 1137 bytes} [Mon Jul 18 16:24:32 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:33.058929 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df11a953-ebe2-49df-a2cf-3c2411dadc6d None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:33.080897 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f7151b-5977-42a2-8b8c-5499814d713b None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:33.114114 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:33.114310 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5d40c1-6bcf-4e96-a039-1735782d84a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:33.114817 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d40c1-6bcf-4e96-a039-1735782d84a6 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:33.115095 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d40c1-6bcf-4e96-a039-1735782d84a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:33.115365 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d40c1-6bcf-4e96-a039-1735782d84a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:33.118651 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5d40c1-6bcf-4e96-a039-1735782d84a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:33.118986 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5d40c1-6bcf-4e96-a039-1735782d84a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:33.122592 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5d40c1-6bcf-4e96-a039-1735782d84a6 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:33.136563 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 256/514] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:33.137863 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5d40c1-6bcf-4e96-a039-1735782d84a6 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:33.143971 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:33.160840 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df11a953-ebe2-49df-a2cf-3c2411dadc6d None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:33.195126 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa196093-8aad-43ea-8752-378dab9a084f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:33.195962 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa196093-8aad-43ea-8752-378dab9a084f None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:33.196255 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa196093-8aad-43ea-8752-378dab9a084f None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:33.196534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa196093-8aad-43ea-8752-378dab9a084f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:33.207168 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa196093-8aad-43ea-8752-378dab9a084f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:33.208684 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa196093-8aad-43ea-8752-378dab9a084f None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:33.225618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-aa196093-8aad-43ea-8752-378dab9a084f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:33.225618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:33.225618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:33.225618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:33.225618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:33.225618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:33.225618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:33.225618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:33.225618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:33.225618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:33.225618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:33.225618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:33.225618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:33.225618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:33.225618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:33.226674 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:33.226674 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:33.226674 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:33.226674 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:33.226674 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:33.226674 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:33.226674 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:33.226674 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:33.226674 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:33.226674 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:33.226674 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:33.226674 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:33.226674 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:33.226674 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:33.226674 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:33.226674 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:33.226674 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:33.227595 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:33.227595 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:33.227595 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:33.227595 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:33.227595 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 259/515] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:33.227855 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa196093-8aad-43ea-8752-378dab9a084f None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:33.248751 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5d40c1-6bcf-4e96-a039-1735782d84a6 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:33.285143 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e48ae4-8a72-4616-87a9-63b478489c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:33.285934 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e48ae4-8a72-4616-87a9-63b478489c6b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:33.286211 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e48ae4-8a72-4616-87a9-63b478489c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:33.286485 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e48ae4-8a72-4616-87a9-63b478489c6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:33.299447 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e48ae4-8a72-4616-87a9-63b478489c6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:33.300951 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e48ae4-8a72-4616-87a9-63b478489c6b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:33.318668 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-a9e48ae4-8a72-4616-87a9-63b478489c6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:33.318668 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:33.318668 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:33.318668 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:33.318668 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:33.318668 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:33.318668 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:33.318668 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:33.318668 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:33.318668 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:33.318668 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:33.318668 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:33.318668 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:33.318668 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:33.318668 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:33.319628 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:33.319628 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:33.319628 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:33.319628 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:33.319628 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:33.319628 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:33.319628 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:33.319628 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:33.319628 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:33.319628 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:33.319628 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:33.319628 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:33.319628 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:33.319628 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:33.319628 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:33.319628 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:33.319628 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:33.320503 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:33.320503 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:33.320503 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:33.320503 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:33.320503 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 257/516] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:33.320767 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e48ae4-8a72-4616-87a9-63b478489c6b None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:33.334222 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa196093-8aad-43ea-8752-378dab9a084f None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:33.344186 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:33.369878 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3067a42-7e9c-4714-857b-e9170064f286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:33.371451 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3067a42-7e9c-4714-857b-e9170064f286 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:33.371767 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3067a42-7e9c-4714-857b-e9170064f286 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:33.372047 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3067a42-7e9c-4714-857b-e9170064f286 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:33.377506 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:33.382123 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3067a42-7e9c-4714-857b-e9170064f286 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:33.383626 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3067a42-7e9c-4714-857b-e9170064f286 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:33.402777 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-b3067a42-7e9c-4714-857b-e9170064f286 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:33.402777 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:33.402777 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:33.402777 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:33.402777 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:33.402777 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:33.402777 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:33.402777 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:33.402777 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:33.402777 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:33.402777 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:33.402777 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:33.402777 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:33.402777 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:33.402777 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:33.404140 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:33.404140 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:33.404140 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:33.404140 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:33.404140 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:33.404140 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:33.404140 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:33.404140 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:33.404140 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:33.404140 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:33.404140 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:33.404140 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:33.404140 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:33.404140 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:33.404140 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:33.404140 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:33.404140 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:33.404987 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:33.404987 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:33.404987 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:33.404987 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:33.404987 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 260/517] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:33.404987 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3067a42-7e9c-4714-857b-e9170064f286 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:33.427408 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e48ae4-8a72-4616-87a9-63b478489c6b None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:33.460873 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6158c43d-e59d-47fc-b7e0-13c4b14c155d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:33.461513 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158c43d-e59d-47fc-b7e0-13c4b14c155d None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:33.461796 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158c43d-e59d-47fc-b7e0-13c4b14c155d None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:33.462072 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158c43d-e59d-47fc-b7e0-13c4b14c155d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:33.505978 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3067a42-7e9c-4714-857b-e9170064f286 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:33.529913 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6158c43d-e59d-47fc-b7e0-13c4b14c155d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:33.531389 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6158c43d-e59d-47fc-b7e0-13c4b14c155d None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:33.545945 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d96b6f-5996-486d-b194-1ae9b96097e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:33.546634 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d96b6f-5996-486d-b194-1ae9b96097e1 None admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:33.546922 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d96b6f-5996-486d-b194-1ae9b96097e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:33.547201 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d96b6f-5996-486d-b194-1ae9b96097e1 None admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:33.547939 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d96b6f-5996-486d-b194-1ae9b96097e1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:33.549384 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d96b6f-5996-486d-b194-1ae9b96097e1 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:33.587261 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:33.613821 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 261/518] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 18 16:24:33 2022] POST /identity/v3/users => generated 247 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:24:33.614630 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d96b6f-5996-486d-b194-1ae9b96097e1 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:33.641368 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 258/519] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:24:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:24:33.641368 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6158c43d-e59d-47fc-b7e0-13c4b14c155d None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:33.649264 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:33.744654 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d96b6f-5996-486d-b194-1ae9b96097e1 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:33.755838 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6158c43d-e59d-47fc-b7e0-13c4b14c155d None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:33.789364 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3739e9d5-3e9f-4611-ad9b-1f00a7b753e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:33.789970 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739e9d5-3e9f-4611-ad9b-1f00a7b753e5 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:33.790287 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739e9d5-3e9f-4611-ad9b-1f00a7b753e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:33.791235 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739e9d5-3e9f-4611-ad9b-1f00a7b753e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:33.791356 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3739e9d5-3e9f-4611-ad9b-1f00a7b753e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:33.791617 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3739e9d5-3e9f-4611-ad9b-1f00a7b753e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:33.794338 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3739e9d5-3e9f-4611-ad9b-1f00a7b753e5 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:33.799538 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02811601-07e9-47c4-afdb-d0b0b5fdd945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:33.800208 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02811601-07e9-47c4-afdb-d0b0b5fdd945 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:33.800496 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02811601-07e9-47c4-afdb-d0b0b5fdd945 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:33.800778 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02811601-07e9-47c4-afdb-d0b0b5fdd945 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:33.801507 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02811601-07e9-47c4-afdb-d0b0b5fdd945 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:33.801813 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02811601-07e9-47c4-afdb-d0b0b5fdd945 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:33.803190 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02811601-07e9-47c4-afdb-d0b0b5fdd945 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:33.806137 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 262/520] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:33.806980 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3739e9d5-3e9f-4611-ad9b-1f00a7b753e5 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:33.821204 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 259/521] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:33.822065 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02811601-07e9-47c4-afdb-d0b0b5fdd945 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:33.844488 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:33.903144 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:33.926337 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3739e9d5-3e9f-4611-ad9b-1f00a7b753e5 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:33.934910 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02811601-07e9-47c4-afdb-d0b0b5fdd945 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:33.962614 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d9be9-2a56-4c3a-af8f-e8f9f8e67cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:33.963289 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d9be9-2a56-4c3a-af8f-e8f9f8e67cc8 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:33.963617 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d9be9-2a56-4c3a-af8f-e8f9f8e67cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:33.964574 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d9be9-2a56-4c3a-af8f-e8f9f8e67cc8 None admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:33.965390 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3d9be9-2a56-4c3a-af8f-e8f9f8e67cc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:33.965610 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3d9be9-2a56-4c3a-af8f-e8f9f8e67cc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:33.966994 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3d9be9-2a56-4c3a-af8f-e8f9f8e67cc8 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:33.974339 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8395baa-0923-477a-a423-b66ad978cb16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:33.975409 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8395baa-0923-477a-a423-b66ad978cb16 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:33.975858 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8395baa-0923-477a-a423-b66ad978cb16 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:33.976287 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8395baa-0923-477a-a423-b66ad978cb16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:33.980004 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 263/522] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:33.981451 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d9be9-2a56-4c3a-af8f-e8f9f8e67cc8 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:33.990200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8395baa-0923-477a-a423-b66ad978cb16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:33.992143 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8395baa-0923-477a-a423-b66ad978cb16 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:34.012531 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-f8395baa-0923-477a-a423-b66ad978cb16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:34.012531 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:34.012531 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:34.012531 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:34.012531 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:34.012531 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:34.012531 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:34.012531 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:34.012531 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:34.012531 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:34.012531 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:34.012531 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:34.012531 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:34.012531 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:34.012531 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:34.013464 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:34.013464 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:34.013464 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:34.013464 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:34.013464 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:34.013464 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:34.013464 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:34.013464 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:34.013464 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:34.013464 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:34.013464 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:34.013464 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:34.013464 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:34.013464 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:34.013464 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:34.013464 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:34.013464 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:34.014316 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:34.014316 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:34.014316 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:34.014316 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:34.014316 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 260/523] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:34.014588 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8395baa-0923-477a-a423-b66ad978cb16 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:34.091200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:34.098355 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d9be9-2a56-4c3a-af8f-e8f9f8e67cc8 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:34.128292 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8395baa-0923-477a-a423-b66ad978cb16 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:34.137368 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25e51c7-6322-4db9-98ca-b0a5b1be9b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:34.138180 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e51c7-6322-4db9-98ca-b0a5b1be9b42 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:34.138459 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e51c7-6322-4db9-98ca-b0a5b1be9b42 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:34.138731 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e51c7-6322-4db9-98ca-b0a5b1be9b42 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:34.141961 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:34.146451 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25e51c7-6322-4db9-98ca-b0a5b1be9b42 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:34.147965 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25e51c7-6322-4db9-98ca-b0a5b1be9b42 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:34.164271 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-e25e51c7-6322-4db9-98ca-b0a5b1be9b42 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:34.164271 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:34.164271 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:34.164271 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:34.164271 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:34.164271 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:34.164271 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:34.164271 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:34.164271 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:34.164271 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:34.164271 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:34.164271 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:34.164271 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:24:34.164271 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:24:34.164271 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:24:34.165389 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:24:34.165389 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:34.165389 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:34.165389 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:34.165389 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:34.165389 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:34.165389 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:34.165389 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:34.165389 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:34.165389 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:34.165389 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:34.165389 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:34.165389 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:34.165389 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:34.165389 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:34.165389 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:34.165389 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:34.166251 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:24:34.166251 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:24:34.166251 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:24:34.166251 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:24:34.166251 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:24:34.166251 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:24:34.166251 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:34.166251 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:34.166251 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 264/524] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:24:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:34.166720 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25e51c7-6322-4db9-98ca-b0a5b1be9b42 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:34.176879 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e1019-8d28-458e-b1ec-b84e8a6744cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:34.177404 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e1019-8d28-458e-b1ec-b84e8a6744cd None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:34.177696 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e1019-8d28-458e-b1ec-b84e8a6744cd None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:34.178073 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e1019-8d28-458e-b1ec-b84e8a6744cd None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:34.187282 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95e1019-8d28-458e-b1ec-b84e8a6744cd None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:34.188803 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95e1019-8d28-458e-b1ec-b84e8a6744cd None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:34.203999 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-d95e1019-8d28-458e-b1ec-b84e8a6744cd None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 16:24:34.203999 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:34.203999 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:34.203999 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:34.203999 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:34.203999 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:34.203999 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:34.203999 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:34.203999 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:34.203999 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:34.203999 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:34.203999 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:34.203999 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:24:34.203999 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:24:34.203999 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:24:34.203999 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:24:34.204929 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:34.204929 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:34.204929 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:24:34.204929 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:34.204929 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:24:34.204929 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:34.204929 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:34.204929 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:34.204929 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:34.204929 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:34.204929 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:34.204929 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:34.204929 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:34.204929 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:34.204929 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:34.204929 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:34.204929 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:34.204929 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:34.205905 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:34.205905 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:34.205905 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:24:34.205905 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:24:34.205905 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:24:34.205905 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:24:34.205905 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:24:34.205905 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:24:34.205905 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 16:24:34.205905 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:34.205905 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 261/525] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 18 16:24:34 2022] GET /identity/v3/users/neutron => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:34.206466 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e1019-8d28-458e-b1ec-b84e8a6744cd None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:34.283050 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25e51c7-6322-4db9-98ca-b0a5b1be9b42 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:34.318164 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e1019-8d28-458e-b1ec-b84e8a6744cd None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:34.319629 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b0c8a4-286a-4770-bf4f-e6b9f1fa39df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:34.320433 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b0c8a4-286a-4770-bf4f-e6b9f1fa39df None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:34.320717 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b0c8a4-286a-4770-bf4f-e6b9f1fa39df None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:34.320988 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b0c8a4-286a-4770-bf4f-e6b9f1fa39df None admin] PATH_INFO: `/v3/projects/service` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:34.328138 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b0c8a4-286a-4770-bf4f-e6b9f1fa39df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:34.329593 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b0c8a4-286a-4770-bf4f-e6b9f1fa39df None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:34.344780 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-e7b0c8a4-286a-4770-bf4f-e6b9f1fa39df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:34.344780 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:34.344780 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:34.344780 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:34.344780 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:34.344780 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:34.344780 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:34.344780 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:34.344780 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:34.344780 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:34.344780 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:34.344780 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:34.344780 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:24:34.344780 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:24:34.344780 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:24:34.345638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:24:34.345638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:34.345638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:34.345638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:34.345638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:34.345638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:34.345638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:34.345638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:34.345638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:34.345638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:34.345638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:34.345638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:34.345638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:34.345638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:34.345638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:34.345638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:34.345638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:34.346506 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:24:34.346506 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:24:34.346506 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:24:34.346506 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:24:34.346506 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:24:34.346506 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:24:34.346506 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:34.346506 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:34.346506 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 265/526] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:24:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:34.346964 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b0c8a4-286a-4770-bf4f-e6b9f1fa39df None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:34.355110 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:34.365117 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace3d7f4-e361-4f7b-9de6-c9cccf496b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:34.365877 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace3d7f4-e361-4f7b-9de6-c9cccf496b79 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:34.366162 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace3d7f4-e361-4f7b-9de6-c9cccf496b79 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:34.367210 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace3d7f4-e361-4f7b-9de6-c9cccf496b79 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:34.367210 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace3d7f4-e361-4f7b-9de6-c9cccf496b79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:34.367657 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace3d7f4-e361-4f7b-9de6-c9cccf496b79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:34.369162 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace3d7f4-e361-4f7b-9de6-c9cccf496b79 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:34.382942 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 262/527] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:34.383823 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace3d7f4-e361-4f7b-9de6-c9cccf496b79 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:34.413249 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:34.463628 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b0c8a4-286a-4770-bf4f-e6b9f1fa39df None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:34.498596 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace3d7f4-e361-4f7b-9de6-c9cccf496b79 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:34.500861 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f64409b-ff65-40a8-b94d-882f071a1189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:34.501633 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f64409b-ff65-40a8-b94d-882f071a1189 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:34.501923 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f64409b-ff65-40a8-b94d-882f071a1189 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:34.502200 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f64409b-ff65-40a8-b94d-882f071a1189 None admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:34.502996 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f64409b-ff65-40a8-b94d-882f071a1189 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:34.503306 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f64409b-ff65-40a8-b94d-882f071a1189 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:34.504772 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f64409b-ff65-40a8-b94d-882f071a1189 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:34.524898 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 266/528] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 18 16:24:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:34.525748 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f64409b-ff65-40a8-b94d-882f071a1189 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:34.540525 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60240e6a-5e32-4733-ba59-4a29cc88d011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:34.541200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60240e6a-5e32-4733-ba59-4a29cc88d011 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:34.541480 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60240e6a-5e32-4733-ba59-4a29cc88d011 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:34.541751 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60240e6a-5e32-4733-ba59-4a29cc88d011 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:34.543253 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60240e6a-5e32-4733-ba59-4a29cc88d011 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:34.543594 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60240e6a-5e32-4733-ba59-4a29cc88d011 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:34.545289 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60240e6a-5e32-4733-ba59-4a29cc88d011 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:34.568398 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 263/529] 10.222.0.26 () {56 vars in 1139 bytes} [Mon Jul 18 16:24:34 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:34.568398 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60240e6a-5e32-4733-ba59-4a29cc88d011 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:34.618008 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:34.653415 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f64409b-ff65-40a8-b94d-882f071a1189 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:34.671657 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:34.678832 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60240e6a-5e32-4733-ba59-4a29cc88d011 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:34.691539 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98758724-eac3-4930-9372-4c29b69bee89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:34.692214 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98758724-eac3-4930-9372-4c29b69bee89 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:34.692503 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98758724-eac3-4930-9372-4c29b69bee89 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:34.692885 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98758724-eac3-4930-9372-4c29b69bee89 None admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:34.693699 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98758724-eac3-4930-9372-4c29b69bee89 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:34.694012 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98758724-eac3-4930-9372-4c29b69bee89 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:34.695462 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98758724-eac3-4930-9372-4c29b69bee89 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:34.708254 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 267/530] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 18 16:24:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:34.712444 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98758724-eac3-4930-9372-4c29b69bee89 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:34.716901 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0572e9f0-022d-42ba-82ad-5f830920fe30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:34.717704 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0572e9f0-022d-42ba-82ad-5f830920fe30 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:34.717993 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0572e9f0-022d-42ba-82ad-5f830920fe30 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:34.718271 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0572e9f0-022d-42ba-82ad-5f830920fe30 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:34.725711 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0572e9f0-022d-42ba-82ad-5f830920fe30 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:34.727173 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0572e9f0-022d-42ba-82ad-5f830920fe30 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:34.742455 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-0572e9f0-022d-42ba-82ad-5f830920fe30 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:34.742455 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:34.742455 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:34.742455 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:34.742455 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:34.742455 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:34.742455 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:34.742455 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:34.742455 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:34.742455 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:34.742455 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:34.742455 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:34.742455 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:24:34.742455 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:24:34.742455 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:24:34.743374 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:24:34.743374 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:34.743374 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:34.743374 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:34.743374 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:34.743374 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:34.743374 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:34.743374 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:34.743374 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:34.743374 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:34.743374 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:34.743374 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:34.743374 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:34.743374 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:34.743374 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:34.743374 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:34.743374 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:34.744257 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:24:34.744257 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:24:34.744257 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:24:34.744257 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:24:34.744257 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:24:34.744257 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:24:34.744257 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:34.744257 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:34.744257 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 264/531] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:24:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:34.746246 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0572e9f0-022d-42ba-82ad-5f830920fe30 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:34.829655 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98758724-eac3-4930-9372-4c29b69bee89 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:34.855601 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0572e9f0-022d-42ba-82ad-5f830920fe30 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:34.859044 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:34.876583 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b17182-24ac-4dc9-b1bf-2fd560514208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:34.876583 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b17182-24ac-4dc9-b1bf-2fd560514208 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:34.876583 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b17182-24ac-4dc9-b1bf-2fd560514208 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:34.876583 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b17182-24ac-4dc9-b1bf-2fd560514208 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:34.876981 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b17182-24ac-4dc9-b1bf-2fd560514208 None admin] RBAC: Adding query filter params (role.id=4fe970e98f394bbfb096c4879bebdf97, scope.project.id=eb9747e2299c4e0991fd7edb4462c7e5, user.id=5b868bfa44b94ca5b470166d39f20e54) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:34.876981 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b17182-24ac-4dc9-b1bf-2fd560514208 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:34.876981 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b17182-24ac-4dc9-b1bf-2fd560514208 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:34.888187 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 268/532] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 18 16:24:34 2022] GET /identity/v3/role_assignments?user.id=5b868bfa44b94ca5b470166d39f20e54&scope.project.id=eb9747e2299c4e0991fd7edb4462c7e5&role.id=4fe970e98f394bbfb096c4879bebdf97 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:34.888187 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b17182-24ac-4dc9-b1bf-2fd560514208 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:34.913038 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054b9fcf-cc60-4e26-b621-732f77f0e4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:34.914113 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b9fcf-cc60-4e26-b621-732f77f0e4fe None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:34.914475 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b9fcf-cc60-4e26-b621-732f77f0e4fe None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:34.915295 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b9fcf-cc60-4e26-b621-732f77f0e4fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:34.933968 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:35.013414 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b17182-24ac-4dc9-b1bf-2fd560514208 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:35.030053 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054b9fcf-cc60-4e26-b621-732f77f0e4fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:35.032286 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054b9fcf-cc60-4e26-b621-732f77f0e4fe None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:35.065082 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf034e9-9f22-4741-8625-8f1a5b2ac55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:35.066535 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf034e9-9f22-4741-8625-8f1a5b2ac55e None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:35.067022 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf034e9-9f22-4741-8625-8f1a5b2ac55e None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:35.067306 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf034e9-9f22-4741-8625-8f1a5b2ac55e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:35.068216 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf034e9-9f22-4741-8625-8f1a5b2ac55e None admin] RBAC: Adding query filter params (role.id=4fe970e98f394bbfb096c4879bebdf97, scope.project.id=eb9747e2299c4e0991fd7edb4462c7e5, user.id=0887908bcb0d4072bf6acb4cbaf1af83) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:35.068544 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf034e9-9f22-4741-8625-8f1a5b2ac55e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:35.070060 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf034e9-9f22-4741-8625-8f1a5b2ac55e None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:35.083634 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 269/533] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 18 16:24:34 2022] GET /identity/v3/role_assignments?user.id=0887908bcb0d4072bf6acb4cbaf1af83&scope.project.id=eb9747e2299c4e0991fd7edb4462c7e5&role.id=4fe970e98f394bbfb096c4879bebdf97 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:35.085300 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf034e9-9f22-4741-8625-8f1a5b2ac55e None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:35.177343 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 265/534] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:24:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:24:35.178705 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054b9fcf-cc60-4e26-b621-732f77f0e4fe None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:35.182300 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:35.307459 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054b9fcf-cc60-4e26-b621-732f77f0e4fe None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:35.358925 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2a75bb-e8af-4b4a-a449-11b959e4127f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:35.359762 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2a75bb-e8af-4b4a-a449-11b959e4127f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:35.360051 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2a75bb-e8af-4b4a-a449-11b959e4127f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:35.360336 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2a75bb-e8af-4b4a-a449-11b959e4127f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:35.374839 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf034e9-9f22-4741-8625-8f1a5b2ac55e None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:35.378250 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:35.388425 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2a75bb-e8af-4b4a-a449-11b959e4127f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:35.389926 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2a75bb-e8af-4b4a-a449-11b959e4127f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:35.418120 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db8e65c-c13b-4925-a5b9-d0b30d4964af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:35.418960 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8e65c-c13b-4925-a5b9-d0b30d4964af None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:35.419631 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8e65c-c13b-4925-a5b9-d0b30d4964af None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:35.420016 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8e65c-c13b-4925-a5b9-d0b30d4964af None admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:35.421916 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db8e65c-c13b-4925-a5b9-d0b30d4964af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:35.422330 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db8e65c-c13b-4925-a5b9-d0b30d4964af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:35.423908 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db8e65c-c13b-4925-a5b9-d0b30d4964af None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:35.432361 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-cf2a75bb-e8af-4b4a-a449-11b959e4127f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:35.432361 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:35.432361 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:35.432361 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:35.432361 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:35.432361 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:35.432361 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:35.432361 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:35.432361 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:35.432361 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:35.432361 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:35.432361 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:35.432361 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:35.432361 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:35.432361 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:35.433520 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:35.433520 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:35.433520 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:35.433520 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:35.433520 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:35.433520 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:35.433520 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:35.433520 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:35.433520 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:35.433520 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:35.433520 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:35.433520 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:35.433520 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:35.433520 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:35.433520 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:35.433520 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:35.433520 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:35.434361 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:35.434361 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:35.434361 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:35.434361 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:35.434361 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 266/535] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:35.442286 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 270/536] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 18 16:24:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:35.443284 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2a75bb-e8af-4b4a-a449-11b959e4127f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:35.448062 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db8e65c-c13b-4925-a5b9-d0b30d4964af None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:35.516425 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:35.562526 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2a75bb-e8af-4b4a-a449-11b959e4127f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:35.601614 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b087478-3fa7-47bb-ac9f-149be70a3d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:35.603020 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b087478-3fa7-47bb-ac9f-149be70a3d51 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:35.603698 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b087478-3fa7-47bb-ac9f-149be70a3d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:35.604163 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b087478-3fa7-47bb-ac9f-149be70a3d51 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:35.606124 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b087478-3fa7-47bb-ac9f-149be70a3d51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:35.606633 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b087478-3fa7-47bb-ac9f-149be70a3d51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:35.608415 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b087478-3fa7-47bb-ac9f-149be70a3d51 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:35.623532 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 267/537] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:35.628794 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b087478-3fa7-47bb-ac9f-149be70a3d51 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:35.747517 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b087478-3fa7-47bb-ac9f-149be70a3d51 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:35.749703 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db8e65c-c13b-4925-a5b9-d0b30d4964af None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:35.765571 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:35.791240 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7578e70a-4d1b-4c21-8476-4839ebb083e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:35.793332 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7578e70a-4d1b-4c21-8476-4839ebb083e5 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:35.794103 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7578e70a-4d1b-4c21-8476-4839ebb083e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:35.794934 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7578e70a-4d1b-4c21-8476-4839ebb083e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:35.795475 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169799b1-d56c-42db-8ff0-4b8199755311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:35.796273 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169799b1-d56c-42db-8ff0-4b8199755311 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:35.796550 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169799b1-d56c-42db-8ff0-4b8199755311 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:35.796830 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169799b1-d56c-42db-8ff0-4b8199755311 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:35.797678 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169799b1-d56c-42db-8ff0-4b8199755311 None admin] RBAC: Adding query filter params (role.id=4fe970e98f394bbfb096c4879bebdf97, scope.project.id=eb9747e2299c4e0991fd7edb4462c7e5, user.id=6f2e55df807c43d7951fa4be67ef5e81) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:35.798051 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169799b1-d56c-42db-8ff0-4b8199755311 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:35.799787 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169799b1-d56c-42db-8ff0-4b8199755311 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:35.805373 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7578e70a-4d1b-4c21-8476-4839ebb083e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:35.806214 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 271/538] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 18 16:24:35 2022] GET /identity/v3/role_assignments?user.id=6f2e55df807c43d7951fa4be67ef5e81&scope.project.id=eb9747e2299c4e0991fd7edb4462c7e5&role.id=4fe970e98f394bbfb096c4879bebdf97 => generated 259 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:35.807403 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7578e70a-4d1b-4c21-8476-4839ebb083e5 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:35.807731 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fece7d8-b21c-4415-b8a7-f79d41c5a302 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:35.807884 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fece7d8-b21c-4415-b8a7-f79d41c5a302 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:35.808130 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fece7d8-b21c-4415-b8a7-f79d41c5a302 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:35.813759 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 272/539] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:35.815278 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e7cfac-cbb3-4d15-b351-03bdeef3f750 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:35.815805 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e7cfac-cbb3-4d15-b351-03bdeef3f750 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:35.816291 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e7cfac-cbb3-4d15-b351-03bdeef3f750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:35.836156 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-7578e70a-4d1b-4c21-8476-4839ebb083e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:35.836156 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:35.836156 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:35.836156 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:35.836156 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:35.836156 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:35.836156 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:35.836156 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:35.836156 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:35.836156 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:35.836156 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:35.836156 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:35.836156 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:24:35.836156 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:24:35.836156 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:24:35.836984 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:24:35.836984 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:35.836984 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:35.836984 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:35.836984 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:35.836984 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:35.836984 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:35.836984 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:35.836984 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:35.836984 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:35.836984 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:35.836984 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:35.836984 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:35.836984 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:35.836984 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:35.836984 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:35.836984 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:35.837829 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:24:35.837829 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:24:35.837829 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:24:35.837829 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:24:35.837829 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:24:35.837829 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:24:35.837829 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:35.837829 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:35.840664 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-07e7cfac-cbb3-4d15-b351-03bdeef3f750 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:35.840853 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 268/540] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:24:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:24:35.842055 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-07e7cfac-cbb3-4d15-b351-03bdeef3f750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:35.848795 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7578e70a-4d1b-4c21-8476-4839ebb083e5 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:35.873423 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:35.971014 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 273/541] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:36.035629 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e7cfac-cbb3-4d15-b351-03bdeef3f750 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:36.042373 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7578e70a-4d1b-4c21-8476-4839ebb083e5 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:36.079960 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde321a-2ff4-4fe5-aab6-040a1aa357ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:36.080974 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde321a-2ff4-4fe5-aab6-040a1aa357ea None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:36.081463 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde321a-2ff4-4fe5-aab6-040a1aa357ea None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:36.081967 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde321a-2ff4-4fe5-aab6-040a1aa357ea None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:36.082967 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebde321a-2ff4-4fe5-aab6-040a1aa357ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:36.083481 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebde321a-2ff4-4fe5-aab6-040a1aa357ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:36.085163 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebde321a-2ff4-4fe5-aab6-040a1aa357ea None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:36.105564 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 269/542] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 18 16:24:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:36.112769 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde321a-2ff4-4fe5-aab6-040a1aa357ea None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:36.144345 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e7cfac-cbb3-4d15-b351-03bdeef3f750 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:36.181201 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6065499-303e-4852-b8d3-bb5380ba08ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:36.182034 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6065499-303e-4852-b8d3-bb5380ba08ee None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:36.182304 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6065499-303e-4852-b8d3-bb5380ba08ee None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:36.182682 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6065499-303e-4852-b8d3-bb5380ba08ee None admin] PATH_INFO: `/v3/roles/service` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:36.195784 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6065499-303e-4852-b8d3-bb5380ba08ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:36.203765 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-f6065499-303e-4852-b8d3-bb5380ba08ee None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:36.203765 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:36.203765 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:36.203765 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:36.203765 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:36.203765 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:36.203765 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:36.203765 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:36.203765 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:36.203765 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:36.203765 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:36.203765 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:36.203765 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:36.203765 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:36.203765 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:36.203765 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:36.204731 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:36.204731 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:36.204731 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:36.204731 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:36.204731 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:36.204731 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:36.204731 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:36.204731 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:36.204731 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:36.204731 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:36.204731 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:36.204731 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:36.204731 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:36.204731 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:36.204731 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:36.204731 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:36.205561 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:36.205561 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:36.205561 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:36.205561 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:36.205561 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:36.205561 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:36.205561 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:36.205561 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:36.205561 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:36.205561 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:36.205561 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:36.205561 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:36.205561 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 274/543] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 18 16:24:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:36.211548 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6065499-303e-4852-b8d3-bb5380ba08ee None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:36.212153 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:36.234251 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:36.322931 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6065499-303e-4852-b8d3-bb5380ba08ee None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:36.359717 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6b9677-3bb4-4faa-8a35-0626ec1b7515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:36.360395 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b9677-3bb4-4faa-8a35-0626ec1b7515 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:36.360705 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b9677-3bb4-4faa-8a35-0626ec1b7515 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:36.360948 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b9677-3bb4-4faa-8a35-0626ec1b7515 None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:36.361698 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6b9677-3bb4-4faa-8a35-0626ec1b7515 None admin] RBAC: Adding query filter params (name=service) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:36.362000 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6b9677-3bb4-4faa-8a35-0626ec1b7515 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:36.363382 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6b9677-3bb4-4faa-8a35-0626ec1b7515 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:36.373410 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 275/544] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 18 16:24:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:36.378756 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6b9677-3bb4-4faa-8a35-0626ec1b7515 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:36.387367 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde321a-2ff4-4fe5-aab6-040a1aa357ea None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:36.451521 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca90673-c3f6-46e2-af24-5f5c702fc16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:36.452651 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca90673-c3f6-46e2-af24-5f5c702fc16b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:36.453180 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca90673-c3f6-46e2-af24-5f5c702fc16b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:36.453681 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca90673-c3f6-46e2-af24-5f5c702fc16b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:36.454731 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca90673-c3f6-46e2-af24-5f5c702fc16b None admin] RBAC: Adding query filter params (role.id=4fe970e98f394bbfb096c4879bebdf97, scope.project.id=eb9747e2299c4e0991fd7edb4462c7e5, user.id=7a6054dbfbbc487794d2a32061b0ac82) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:36.455252 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca90673-c3f6-46e2-af24-5f5c702fc16b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:36.456958 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca90673-c3f6-46e2-af24-5f5c702fc16b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:36.463628 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 270/545] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 18 16:24:36 2022] GET /identity/v3/role_assignments?user.id=7a6054dbfbbc487794d2a32061b0ac82&scope.project.id=eb9747e2299c4e0991fd7edb4462c7e5&role.id=4fe970e98f394bbfb096c4879bebdf97 => generated 259 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:36.467231 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:36.491042 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6b9677-3bb4-4faa-8a35-0626ec1b7515 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:36.533927 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf89dc3-b0c0-4e1d-afc2-179d1befbbdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:36.534762 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf89dc3-b0c0-4e1d-afc2-179d1befbbdd None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:36.535046 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf89dc3-b0c0-4e1d-afc2-179d1befbbdd None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:36.535325 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf89dc3-b0c0-4e1d-afc2-179d1befbbdd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:36.548737 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf89dc3-b0c0-4e1d-afc2-179d1befbbdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:36.550241 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf89dc3-b0c0-4e1d-afc2-179d1befbbdd None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:36.568894 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-bcf89dc3-b0c0-4e1d-afc2-179d1befbbdd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:36.568894 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:36.568894 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:36.568894 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:36.568894 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:36.568894 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:36.568894 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:36.568894 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:36.568894 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:36.568894 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:36.568894 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:36.568894 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:36.568894 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:36.568894 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:36.568894 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:36.569837 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:36.569837 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:36.569837 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:36.569837 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:36.569837 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:36.569837 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:36.569837 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:36.569837 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:36.569837 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:36.569837 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:36.569837 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:36.569837 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:36.569837 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:36.569837 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:36.569837 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:36.569837 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:36.569837 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:36.570760 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:36.570760 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:36.570760 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:36.570760 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:36.570760 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 276/546] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:24:36.576316 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca90673-c3f6-46e2-af24-5f5c702fc16b None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:36.731697 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:36.735401 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca90673-c3f6-46e2-af24-5f5c702fc16b None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:36.780245 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59827558-8d49-4f5e-ad6f-a2559cbb7670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:36.780918 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59827558-8d49-4f5e-ad6f-a2559cbb7670 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:36.781207 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59827558-8d49-4f5e-ad6f-a2559cbb7670 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:36.781485 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59827558-8d49-4f5e-ad6f-a2559cbb7670 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:36.782216 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59827558-8d49-4f5e-ad6f-a2559cbb7670 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:36.782523 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59827558-8d49-4f5e-ad6f-a2559cbb7670 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:36.783924 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59827558-8d49-4f5e-ad6f-a2559cbb7670 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:36.803412 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 271/547] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:36.808902 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf89dc3-b0c0-4e1d-afc2-179d1befbbdd None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:36.918116 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf89dc3-b0c0-4e1d-afc2-179d1befbbdd None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:36.949980 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:36.955764 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b22116-caad-482b-abfe-6b72c7d93e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:36.956653 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b22116-caad-482b-abfe-6b72c7d93e73 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:36.956938 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b22116-caad-482b-abfe-6b72c7d93e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:36.957216 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b22116-caad-482b-abfe-6b72c7d93e73 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:36.965948 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b22116-caad-482b-abfe-6b72c7d93e73 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:36.967466 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b22116-caad-482b-abfe-6b72c7d93e73 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:36.983228 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-30b22116-caad-482b-abfe-6b72c7d93e73 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 16:24:36.983228 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:36.983228 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:36.983228 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:36.983228 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:36.983228 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:36.983228 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:36.983228 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:36.983228 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:36.983228 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:36.983228 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:36.983228 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:36.983228 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:24:36.983228 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:24:36.983228 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:24:36.983228 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:24:36.984288 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:36.984288 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:36.984288 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:24:36.984288 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:36.984288 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:24:36.984288 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:36.984288 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:36.984288 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:36.984288 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:36.984288 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:36.984288 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:36.984288 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:36.984288 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:36.984288 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:36.984288 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:36.984288 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:36.984288 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:36.984288 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:36.985190 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:36.985190 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:36.985190 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:24:36.985190 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:24:36.985190 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:24:36.985190 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:24:36.985190 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:24:36.985190 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:24:36.985190 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 16:24:36.985190 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:36.985190 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 277/548] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 18 16:24:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:24:36.990313 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59827558-8d49-4f5e-ad6f-a2559cbb7670 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:37.122522 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59827558-8d49-4f5e-ad6f-a2559cbb7670 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:37.165121 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1335d821-56ae-4b99-899b-53ca534b91c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:37.165805 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335d821-56ae-4b99-899b-53ca534b91c5 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:37.166091 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335d821-56ae-4b99-899b-53ca534b91c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:37.166368 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335d821-56ae-4b99-899b-53ca534b91c5 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:37.167239 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1335d821-56ae-4b99-899b-53ca534b91c5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:37.167568 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1335d821-56ae-4b99-899b-53ca534b91c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:37.169149 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1335d821-56ae-4b99-899b-53ca534b91c5 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:37.190958 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 272/549] 10.222.0.26 () {56 vars in 1135 bytes} [Mon Jul 18 16:24:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:37.196559 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b22116-caad-482b-abfe-6b72c7d93e73 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:37.314386 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b22116-caad-482b-abfe-6b72c7d93e73 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:37.351557 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dc40ea-754c-43a9-bb77-80ec9626e539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:37.352276 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc40ea-754c-43a9-bb77-80ec9626e539 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:37.352562 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc40ea-754c-43a9-bb77-80ec9626e539 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:37.352839 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc40ea-754c-43a9-bb77-80ec9626e539 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:37.372193 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd50e72b-61ac-440e-8a5c-8a032af51e76 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:37.372819 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd50e72b-61ac-440e-8a5c-8a032af51e76 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:37.373055 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd50e72b-61ac-440e-8a5c-8a032af51e76 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:37.375367 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 273/550] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:37.381281 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff451d2-c57c-406c-b6b1-3c30ee42fa51 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:37.381704 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff451d2-c57c-406c-b6b1-3c30ee42fa51 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:37.381981 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff451d2-c57c-406c-b6b1-3c30ee42fa51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:37.403175 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-fff451d2-c57c-406c-b6b1-3c30ee42fa51 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:37.405452 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-fff451d2-c57c-406c-b6b1-3c30ee42fa51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:37.422271 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:37.451158 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:37.461969 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dc40ea-754c-43a9-bb77-80ec9626e539 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:37.463450 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dc40ea-754c-43a9-bb77-80ec9626e539 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:37.530083 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 274/551] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:37.570653 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff451d2-c57c-406c-b6b1-3c30ee42fa51 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:37.571112 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 278/552] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:24:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:24:37.579567 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dc40ea-754c-43a9-bb77-80ec9626e539 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:37.684921 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:37.694244 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dc40ea-754c-43a9-bb77-80ec9626e539 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:37.733101 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ce5b24-3d20-4470-bb66-98a28db490cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:37.733935 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce5b24-3d20-4470-bb66-98a28db490cf None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:37.734219 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce5b24-3d20-4470-bb66-98a28db490cf None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:37.734494 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce5b24-3d20-4470-bb66-98a28db490cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:37.746275 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ce5b24-3d20-4470-bb66-98a28db490cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:37.747778 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ce5b24-3d20-4470-bb66-98a28db490cf None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:37.766295 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-f5ce5b24-3d20-4470-bb66-98a28db490cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:37.766295 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:37.766295 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:37.766295 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:37.766295 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:37.766295 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:37.766295 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:37.766295 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:37.766295 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:37.766295 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:37.766295 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:37.766295 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:37.766295 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:37.766295 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:37.766295 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:37.767219 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:37.767219 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:37.767219 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:37.767219 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:37.767219 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:37.767219 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:37.767219 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:37.767219 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:37.767219 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:37.767219 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:37.767219 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:37.767219 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:37.767219 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:37.767219 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:37.767219 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:37.767219 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:37.767219 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:37.768183 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:37.768183 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:37.768183 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:37.768183 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:37.768183 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 279/553] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:24:37.770804 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97629cd-728c-4e19-a081-5d4c733c8364 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:37.771093 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97629cd-728c-4e19-a081-5d4c733c8364 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:37.771376 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97629cd-728c-4e19-a081-5d4c733c8364 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:37.773143 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 280/554] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:37.773853 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97629cd-728c-4e19-a081-5d4c733c8364 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:37.813869 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff451d2-c57c-406c-b6b1-3c30ee42fa51 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:37.861760 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f297d88e-f9f3-413f-b36a-4277c52b6aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:37.862603 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297d88e-f9f3-413f-b36a-4277c52b6aee None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:37.862901 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297d88e-f9f3-413f-b36a-4277c52b6aee None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:37.863159 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297d88e-f9f3-413f-b36a-4277c52b6aee None admin] PATH_INFO: `/v3/roles/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:37.878730 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f297d88e-f9f3-413f-b36a-4277c52b6aee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:37.887408 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-f297d88e-f9f3-413f-b36a-4277c52b6aee None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:37.887408 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:37.887408 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:37.887408 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:37.887408 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:37.887408 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:37.887408 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:37.887408 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:37.887408 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:37.887408 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:37.887408 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:37.887408 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:37.887408 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:37.887408 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:37.887408 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:37.887408 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:37.888365 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:37.888365 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:37.888365 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:37.888365 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:37.888365 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:37.888365 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:37.888365 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:37.888365 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:37.888365 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:37.888365 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:37.888365 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:37.888365 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:37.888365 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:37.888365 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:37.888365 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:37.888365 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:37.889155 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:37.889155 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:37.889155 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:37.889155 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:37.889155 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:37.889155 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:37.889155 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:37.889155 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:37.889155 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:37.889155 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:37.889155 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:37.889155 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:37.889155 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 275/555] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 18 16:24:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:37.890398 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c49c7-3ea9-42af-b3a7-842ae4a57c11 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:37.890492 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c49c7-3ea9-42af-b3a7-842ae4a57c11 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:37.890777 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c49c7-3ea9-42af-b3a7-842ae4a57c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:37.904617 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97629cd-728c-4e19-a081-5d4c733c8364 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:37.908644 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-509c49c7-3ea9-42af-b3a7-842ae4a57c11 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:37.909851 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-509c49c7-3ea9-42af-b3a7-842ae4a57c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:37.912170 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:37.943411 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf82f38-e8a5-43f3-aa13-f86fa295a312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:37.946377 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf82f38-e8a5-43f3-aa13-f86fa295a312 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:37.947198 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf82f38-e8a5-43f3-aa13-f86fa295a312 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:37.947717 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf82f38-e8a5-43f3-aa13-f86fa295a312 None admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:37.948910 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf82f38-e8a5-43f3-aa13-f86fa295a312 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:37.949447 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf82f38-e8a5-43f3-aa13-f86fa295a312 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:37.952036 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf82f38-e8a5-43f3-aa13-f86fa295a312 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:37.970460 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 281/556] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:37.971790 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf82f38-e8a5-43f3-aa13-f86fa295a312 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:37.975721 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:37.988574 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 276/557] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:37.997064 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509c49c7-3ea9-42af-b3a7-842ae4a57c11 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:38.083652 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf82f38-e8a5-43f3-aa13-f86fa295a312 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:38.123137 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d45919-f9a7-49a5-a1bb-0d2286177e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:38.124072 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d45919-f9a7-49a5-a1bb-0d2286177e3a None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:38.124391 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d45919-f9a7-49a5-a1bb-0d2286177e3a None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:38.124761 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d45919-f9a7-49a5-a1bb-0d2286177e3a None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:38.125706 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d45919-f9a7-49a5-a1bb-0d2286177e3a None admin] RBAC: Adding query filter params (name=service) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:38.126110 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d45919-f9a7-49a5-a1bb-0d2286177e3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:38.127772 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d45919-f9a7-49a5-a1bb-0d2286177e3a None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:38.129913 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509c49c7-3ea9-42af-b3a7-842ae4a57c11 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:38.137785 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 282/558] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 18 16:24:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:38.139029 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d45919-f9a7-49a5-a1bb-0d2286177e3a None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:38.169704 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f60dde-243b-4b33-900e-a3f24ce4ff88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:38.170958 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f60dde-243b-4b33-900e-a3f24ce4ff88 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:38.171464 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f60dde-243b-4b33-900e-a3f24ce4ff88 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:38.171911 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f60dde-243b-4b33-900e-a3f24ce4ff88 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:38.179879 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f60dde-243b-4b33-900e-a3f24ce4ff88 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:38.181864 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f60dde-243b-4b33-900e-a3f24ce4ff88 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:38.184142 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:38.197802 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-12f60dde-243b-4b33-900e-a3f24ce4ff88 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:38.197802 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:38.197802 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:38.197802 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:38.197802 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:38.197802 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:38.197802 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:38.197802 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:38.197802 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:38.197802 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:38.197802 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:38.197802 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:38.197802 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:24:38.197802 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:24:38.197802 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:24:38.199336 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:24:38.199336 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:38.199336 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:38.199336 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:38.199336 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:38.199336 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:38.199336 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:38.199336 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:38.199336 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:38.199336 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:38.199336 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:38.199336 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:38.199336 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:38.199336 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:38.199336 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:38.199336 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:38.199336 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:38.200395 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:24:38.200395 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:24:38.200395 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:24:38.200395 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:24:38.200395 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:24:38.200395 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:24:38.200395 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:38.200395 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:38.202472 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 277/559] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:24:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:38.203629 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f60dde-243b-4b33-900e-a3f24ce4ff88 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:38.221299 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:38.262748 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d45919-f9a7-49a5-a1bb-0d2286177e3a None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:38.302587 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf9af1-d81f-4218-b81f-902976dd8baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:38.303442 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf9af1-d81f-4218-b81f-902976dd8baa None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:38.303757 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf9af1-d81f-4218-b81f-902976dd8baa None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:38.304075 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf9af1-d81f-4218-b81f-902976dd8baa None admin] PATH_INFO: `/v3/roles/service` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:38.317698 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabf9af1-d81f-4218-b81f-902976dd8baa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:38.324914 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f60dde-243b-4b33-900e-a3f24ce4ff88 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:38.327304 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-aabf9af1-d81f-4218-b81f-902976dd8baa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:38.327304 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:38.327304 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:38.327304 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:38.327304 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:38.327304 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:38.327304 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:38.327304 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:38.327304 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:38.327304 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:38.327304 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:38.327304 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:38.327304 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:38.327304 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:38.327304 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:38.327304 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:38.328317 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:38.328317 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:38.328317 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:38.328317 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:38.328317 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:38.328317 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:38.328317 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:38.328317 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:38.328317 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:38.328317 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:38.328317 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:38.328317 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:38.328317 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:38.328317 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:38.328317 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:38.328317 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:38.329131 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:38.329131 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:38.329131 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:38.329131 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:38.329131 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:38.329131 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:38.329131 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:38.329131 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:38.329131 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:38.329131 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:38.329131 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:38.329131 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:38.329131 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 283/560] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 18 16:24:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:38.329812 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf9af1-d81f-4218-b81f-902976dd8baa None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:38.381076 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c91b1-0e3f-46f4-926f-71079030b3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:38.382045 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c91b1-0e3f-46f4-926f-71079030b3ed None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:38.382453 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c91b1-0e3f-46f4-926f-71079030b3ed None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:38.382862 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c91b1-0e3f-46f4-926f-71079030b3ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:38.394417 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465c91b1-0e3f-46f4-926f-71079030b3ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:38.396112 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465c91b1-0e3f-46f4-926f-71079030b3ed None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:38.414829 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-465c91b1-0e3f-46f4-926f-71079030b3ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:38.414829 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:38.414829 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:38.414829 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:38.414829 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:38.414829 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:38.414829 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:38.414829 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:38.414829 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:38.414829 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:38.414829 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:38.414829 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:38.414829 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:38.414829 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:38.414829 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:38.416015 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:38.416015 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:38.416015 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:38.416015 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:38.416015 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:38.416015 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:38.416015 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:38.416015 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:38.416015 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:38.416015 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:38.416015 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:38.416015 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:38.416015 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:38.416015 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:38.416015 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:38.416015 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:38.416015 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:38.417687 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:38.417687 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:38.417687 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:38.417687 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:38.419335 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 278/561] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:38.420553 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c91b1-0e3f-46f4-926f-71079030b3ed None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:38.479420 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:38.485528 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf9af1-d81f-4218-b81f-902976dd8baa None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:38.518231 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:38.527825 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05d254b-d23c-4a2c-9530-3827a02d5399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:38.528498 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05d254b-d23c-4a2c-9530-3827a02d5399 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:38.528790 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05d254b-d23c-4a2c-9530-3827a02d5399 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:38.529047 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05d254b-d23c-4a2c-9530-3827a02d5399 None admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:38.529841 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05d254b-d23c-4a2c-9530-3827a02d5399 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:38.530158 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05d254b-d23c-4a2c-9530-3827a02d5399 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:38.531636 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05d254b-d23c-4a2c-9530-3827a02d5399 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:38.534007 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c91b1-0e3f-46f4-926f-71079030b3ed None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:38.544018 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 284/562] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 18 16:24:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:38.544018 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05d254b-d23c-4a2c-9530-3827a02d5399 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:38.574483 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3959d69d-e491-42c2-b827-b870beec5c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:38.575161 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3959d69d-e491-42c2-b827-b870beec5c98 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:38.575529 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3959d69d-e491-42c2-b827-b870beec5c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:38.575927 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3959d69d-e491-42c2-b827-b870beec5c98 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:38.576694 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3959d69d-e491-42c2-b827-b870beec5c98 None admin] RBAC: Adding query filter params (name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:38.577119 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3959d69d-e491-42c2-b827-b870beec5c98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:38.578561 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3959d69d-e491-42c2-b827-b870beec5c98 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:38.591803 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 279/563] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 18 16:24:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:38.591803 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b246650-a5cd-43a1-97bb-cbc5579a0896 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:38.591803 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b246650-a5cd-43a1-97bb-cbc5579a0896 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:38.591803 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b246650-a5cd-43a1-97bb-cbc5579a0896 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:38.591803 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 280/564] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:38.592546 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b246650-a5cd-43a1-97bb-cbc5579a0896 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:38.685446 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05d254b-d23c-4a2c-9530-3827a02d5399 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:38.723630 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b246650-a5cd-43a1-97bb-cbc5579a0896 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:38.724525 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f703e7-d43f-48b8-8925-524233959274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:38.725202 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f703e7-d43f-48b8-8925-524233959274 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:38.725487 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f703e7-d43f-48b8-8925-524233959274 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:38.725764 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f703e7-d43f-48b8-8925-524233959274 None admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:38.726500 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f703e7-d43f-48b8-8925-524233959274 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:38.726809 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f703e7-d43f-48b8-8925-524233959274 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:38.728221 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f703e7-d43f-48b8-8925-524233959274 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:38.738890 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:38.746386 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 285/565] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:38.746386 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f703e7-d43f-48b8-8925-524233959274 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:38.765685 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dab590e-6637-4ef6-8564-bda8cdd21b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:38.766357 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab590e-6637-4ef6-8564-bda8cdd21b96 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:38.766642 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab590e-6637-4ef6-8564-bda8cdd21b96 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:38.766923 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab590e-6637-4ef6-8564-bda8cdd21b96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:38.767794 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dab590e-6637-4ef6-8564-bda8cdd21b96 None admin] RBAC: Adding query filter params (role.id=4fe970e98f394bbfb096c4879bebdf97, scope.project.id=eb9747e2299c4e0991fd7edb4462c7e5, user.id=43854d7a6df2466eb7976fabe6011767) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:38.768107 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dab590e-6637-4ef6-8564-bda8cdd21b96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:38.770070 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dab590e-6637-4ef6-8564-bda8cdd21b96 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:38.777762 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 281/566] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 18 16:24:38 2022] GET /identity/v3/role_assignments?user.id=43854d7a6df2466eb7976fabe6011767&scope.project.id=eb9747e2299c4e0991fd7edb4462c7e5&role.id=4fe970e98f394bbfb096c4879bebdf97 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:38.779365 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae636a60-c83b-4ee7-8922-c611ac0deff5 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:38.779679 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae636a60-c83b-4ee7-8922-c611ac0deff5 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:38.779954 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae636a60-c83b-4ee7-8922-c611ac0deff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:38.799627 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-ae636a60-c83b-4ee7-8922-c611ac0deff5 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:38.801185 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-ae636a60-c83b-4ee7-8922-c611ac0deff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:38.812807 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:38.879987 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 282/567] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:38.880832 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae636a60-c83b-4ee7-8922-c611ac0deff5 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:38.902491 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f703e7-d43f-48b8-8925-524233959274 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:38.949838 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f4420f-a545-4361-a409-ee9de34476ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:38.950657 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f4420f-a545-4361-a409-ee9de34476ed None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:38.950941 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f4420f-a545-4361-a409-ee9de34476ed None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:38.951219 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f4420f-a545-4361-a409-ee9de34476ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:38.963029 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f4420f-a545-4361-a409-ee9de34476ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:38.964572 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f4420f-a545-4361-a409-ee9de34476ed None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:38.981794 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-d6f4420f-a545-4361-a409-ee9de34476ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:38.981794 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:38.981794 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:38.981794 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:38.981794 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:38.981794 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:38.981794 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:38.981794 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:38.981794 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:38.981794 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:38.981794 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:38.981794 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:38.981794 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:38.981794 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:38.981794 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:38.982960 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:38.982960 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:38.982960 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:38.982960 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:38.982960 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:38.982960 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:38.982960 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:38.982960 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:38.982960 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:38.982960 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:38.982960 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:38.982960 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:38.982960 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:38.982960 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:38.982960 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:38.982960 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:38.982960 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:38.983977 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:38.983977 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:38.983977 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:38.983977 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:38.983977 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 286/568] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:38.984679 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a3c51-e010-4c75-8eb0-7f928deaec6f None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:38.984963 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a3c51-e010-4c75-8eb0-7f928deaec6f None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:38.985625 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a3c51-e010-4c75-8eb0-7f928deaec6f None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:38.990846 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 287/569] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:38.991771 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4a3c51-e010-4c75-8eb0-7f928deaec6f None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:38.999570 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae636a60-c83b-4ee7-8922-c611ac0deff5 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:39.008854 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:39.038237 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0948a27-b5bd-4c74-9a90-70b5f33ea7c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:39.039036 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0948a27-b5bd-4c74-9a90-70b5f33ea7c8 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:39.039321 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0948a27-b5bd-4c74-9a90-70b5f33ea7c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:39.039644 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0948a27-b5bd-4c74-9a90-70b5f33ea7c8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:39.047675 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0948a27-b5bd-4c74-9a90-70b5f33ea7c8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:39.049695 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0948a27-b5bd-4c74-9a90-70b5f33ea7c8 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:39.068426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-a0948a27-b5bd-4c74-9a90-70b5f33ea7c8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:24:39.068426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:39.068426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:39.068426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:39.068426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:39.068426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:39.068426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:39.068426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:39.068426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:39.068426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:39.068426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:39.068426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:39.068426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:24:39.068426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:24:39.068426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:24:39.068426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:24:39.069407 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:39.069407 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:39.069407 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:24:39.069407 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:39.069407 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:24:39.069407 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:39.069407 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:39.069407 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:39.069407 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:39.069407 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:39.069407 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:39.069407 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:39.069407 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:39.069407 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:39.069407 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:39.069407 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:39.069407 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:39.069407 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:39.070320 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:39.070320 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:39.070320 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:24:39.070320 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:24:39.070320 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:24:39.070320 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:24:39.070320 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:24:39.070320 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:24:39.070320 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:24:39.070320 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:39.071430 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 283/570] 10.222.0.26 () {56 vars in 1093 bytes} [Mon Jul 18 16:24:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:39.071762 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0948a27-b5bd-4c74-9a90-70b5f33ea7c8 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:39.106138 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:39.111273 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4a3c51-e010-4c75-8eb0-7f928deaec6f None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:39.148098 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145961c8-2459-4f72-833e-a5e486dffabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:39.148907 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145961c8-2459-4f72-833e-a5e486dffabe None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:39.149188 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145961c8-2459-4f72-833e-a5e486dffabe None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:39.149464 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145961c8-2459-4f72-833e-a5e486dffabe None admin] PATH_INFO: `/v3/roles/service` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:39.160423 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145961c8-2459-4f72-833e-a5e486dffabe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:39.169179 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-145961c8-2459-4f72-833e-a5e486dffabe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:39.169179 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:39.169179 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:39.169179 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:39.169179 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:39.169179 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:39.169179 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:39.169179 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:39.169179 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:39.169179 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:39.169179 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:39.169179 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:39.169179 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:39.169179 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:39.169179 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:39.169179 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:39.170331 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:39.170331 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:39.170331 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:39.170331 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:39.170331 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:39.170331 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:39.170331 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:39.170331 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:39.170331 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:39.170331 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:39.170331 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:39.170331 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:39.170331 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:39.170331 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:39.170331 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:39.170331 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:39.171148 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:39.171148 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:39.171148 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:39.171148 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:39.171148 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:39.171148 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:39.171148 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:39.171148 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:39.171148 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:39.171148 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:39.171148 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:39.171148 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:39.171148 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 288/571] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 18 16:24:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:39.172462 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17701cb0-95eb-4d38-9380-e2aa30af9931 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:39.172462 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17701cb0-95eb-4d38-9380-e2aa30af9931 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:39.172591 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17701cb0-95eb-4d38-9380-e2aa30af9931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:39.182482 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0948a27-b5bd-4c74-9a90-70b5f33ea7c8 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:39.189515 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-17701cb0-95eb-4d38-9380-e2aa30af9931 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:39.190845 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-17701cb0-95eb-4d38-9380-e2aa30af9931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:39.220189 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ccf09a-f6a0-44f3-89b1-0627a63cb77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:39.220867 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ccf09a-f6a0-44f3-89b1-0627a63cb77f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:39.221161 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ccf09a-f6a0-44f3-89b1-0627a63cb77f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:39.221444 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ccf09a-f6a0-44f3-89b1-0627a63cb77f None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:39.222180 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ccf09a-f6a0-44f3-89b1-0627a63cb77f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:39.222491 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ccf09a-f6a0-44f3-89b1-0627a63cb77f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:39.223903 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ccf09a-f6a0-44f3-89b1-0627a63cb77f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:39.236871 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 284/572] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:39.237055 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ccf09a-f6a0-44f3-89b1-0627a63cb77f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:39.265194 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:39.275668 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 289/573] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:39.275668 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17701cb0-95eb-4d38-9380-e2aa30af9931 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:39.354015 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ccf09a-f6a0-44f3-89b1-0627a63cb77f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:39.379359 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:39.390081 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b896fa1c-93d1-40b6-a2c6-fa5873f88faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:39.403406 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b896fa1c-93d1-40b6-a2c6-fa5873f88faa None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:39.403406 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b896fa1c-93d1-40b6-a2c6-fa5873f88faa None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:39.403406 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b896fa1c-93d1-40b6-a2c6-fa5873f88faa None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:39.403406 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b896fa1c-93d1-40b6-a2c6-fa5873f88faa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:39.403406 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b896fa1c-93d1-40b6-a2c6-fa5873f88faa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:39.403809 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17701cb0-95eb-4d38-9380-e2aa30af9931 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:39.403894 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b896fa1c-93d1-40b6-a2c6-fa5873f88faa None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:39.412364 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 285/574] 10.222.0.26 () {56 vars in 1133 bytes} [Mon Jul 18 16:24:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:39.413441 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b896fa1c-93d1-40b6-a2c6-fa5873f88faa None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:39.430558 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf23ca-361f-4c34-a7dc-859e6036db18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:39.431243 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf23ca-361f-4c34-a7dc-859e6036db18 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:39.431550 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf23ca-361f-4c34-a7dc-859e6036db18 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:39.431834 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf23ca-361f-4c34-a7dc-859e6036db18 None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:39.432583 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bf23ca-361f-4c34-a7dc-859e6036db18 None admin] RBAC: Adding query filter params (name=service) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:39.432936 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bf23ca-361f-4c34-a7dc-859e6036db18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:39.434337 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bf23ca-361f-4c34-a7dc-859e6036db18 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:39.447011 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 290/575] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 18 16:24:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:39.447011 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf23ca-361f-4c34-a7dc-859e6036db18 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:39.511350 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:39.528071 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b896fa1c-93d1-40b6-a2c6-fa5873f88faa None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:39.551002 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf23ca-361f-4c34-a7dc-859e6036db18 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:39.565595 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b2039-5b07-4ca6-a891-9edda0cf66c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:39.566386 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b2039-5b07-4ca6-a891-9edda0cf66c0 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:39.566683 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b2039-5b07-4ca6-a891-9edda0cf66c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:39.566944 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b2039-5b07-4ca6-a891-9edda0cf66c0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:39.575622 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587b2039-5b07-4ca6-a891-9edda0cf66c0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:39.576725 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587b2039-5b07-4ca6-a891-9edda0cf66c0 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:39.591935 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f839558-5b36-4110-bd46-c62f84b8ffac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:39.593562 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f839558-5b36-4110-bd46-c62f84b8ffac None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:39.593562 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f839558-5b36-4110-bd46-c62f84b8ffac None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:39.593562 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f839558-5b36-4110-bd46-c62f84b8ffac None admin] PATH_INFO: `/v3/roles/service` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:39.595938 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-587b2039-5b07-4ca6-a891-9edda0cf66c0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 18 16:24:39.595938 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:39.595938 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:39.595938 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:39.595938 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:39.595938 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:39.595938 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:39.595938 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:39.595938 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:39.595938 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:39.595938 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:39.595938 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:39.595938 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:24:39.595938 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:24:39.595938 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:24:39.595938 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:24:39.597005 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:39.597005 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:39.597005 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:24:39.597005 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:39.597005 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:24:39.597005 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:39.597005 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:39.597005 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:39.597005 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:39.597005 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:39.597005 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:39.597005 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:39.597005 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:39.597005 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:39.597005 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:39.597005 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:39.597005 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:39.597005 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:39.597978 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:39.597978 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:39.597978 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:24:39.597978 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:24:39.597978 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:24:39.597978 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:24:39.597978 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:24:39.597978 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:24:39.597978 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 18 16:24:39.597978 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:39.597978 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 286/576] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:24:39 2022] GET /identity/v3/users/glance => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:39.597978 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b2039-5b07-4ca6-a891-9edda0cf66c0 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:39.610949 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f839558-5b36-4110-bd46-c62f84b8ffac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:39.618665 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-6f839558-5b36-4110-bd46-c62f84b8ffac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:39.618665 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:39.618665 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:39.618665 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:39.618665 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:39.618665 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:39.618665 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:39.618665 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:39.618665 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:39.618665 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:39.618665 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:39.618665 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:39.618665 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:39.618665 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:39.618665 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:39.618665 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:39.619850 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:39.619850 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:39.619850 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:39.619850 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:39.619850 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:39.619850 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:39.619850 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:39.619850 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:39.619850 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:39.619850 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:39.619850 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:39.619850 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:39.619850 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:39.619850 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:39.619850 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:39.619850 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:39.626215 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:39.626215 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:39.626215 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:39.626215 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:39.626215 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:39.626215 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:39.626215 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:39.626215 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:39.626215 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:39.626215 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:39.626215 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:39.626215 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:39.626215 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 291/577] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 18 16:24:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:39.626215 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f839558-5b36-4110-bd46-c62f84b8ffac None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:39.637641 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:39.715741 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b2039-5b07-4ca6-a891-9edda0cf66c0 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:39.731387 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f839558-5b36-4110-bd46-c62f84b8ffac None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:39.754649 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eac8a06-b0a5-4c43-a3ca-78289b7d08a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:39.755439 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac8a06-b0a5-4c43-a3ca-78289b7d08a7 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:39.755776 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac8a06-b0a5-4c43-a3ca-78289b7d08a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:39.756055 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac8a06-b0a5-4c43-a3ca-78289b7d08a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:39.767974 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eac8a06-b0a5-4c43-a3ca-78289b7d08a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:39.768699 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa4d346-2c91-4652-97a5-327a0fedb78b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:39.769433 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eac8a06-b0a5-4c43-a3ca-78289b7d08a7 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:39.769707 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa4d346-2c91-4652-97a5-327a0fedb78b None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:39.770103 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa4d346-2c91-4652-97a5-327a0fedb78b None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:39.770478 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa4d346-2c91-4652-97a5-327a0fedb78b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:39.772576 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:39.786351 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa4d346-2c91-4652-97a5-327a0fedb78b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:39.786351 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa4d346-2c91-4652-97a5-327a0fedb78b None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:39.791067 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-4eac8a06-b0a5-4c43-a3ca-78289b7d08a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:39.791067 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:39.791067 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:39.791067 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:39.791067 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:39.791067 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:39.791067 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:39.791067 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:39.791067 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:39.791067 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:39.791067 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:39.791067 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:39.791067 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:39.791067 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:39.791067 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:39.792069 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:39.792069 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:39.792069 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:39.792069 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:39.792069 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:39.792069 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:39.792069 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:39.792069 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:39.792069 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:39.792069 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:39.792069 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:39.792069 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:39.792069 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:39.792069 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:39.792069 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:39.792069 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:39.792069 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:39.793192 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:39.793192 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:39.793192 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:39.793192 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:39.797257 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 287/578] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:39.798227 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eac8a06-b0a5-4c43-a3ca-78289b7d08a7 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:39.805266 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-bfa4d346-2c91-4652-97a5-327a0fedb78b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:39.805266 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:39.805266 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:39.805266 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:39.805266 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:39.805266 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:39.805266 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:39.805266 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:39.805266 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:39.805266 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:39.805266 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:39.805266 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:39.805266 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:39.805266 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:39.805266 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:39.806339 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:39.806339 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:39.806339 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:39.806339 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:39.806339 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:39.806339 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:39.806339 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:39.806339 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:39.806339 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:39.806339 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:39.806339 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:39.806339 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:39.806339 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:39.806339 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:39.806339 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:39.806339 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:39.806339 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:39.807334 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:39.807334 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:39.807334 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:39.807334 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:39.814756 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 292/579] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:39.814756 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa4d346-2c91-4652-97a5-327a0fedb78b None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:39.881533 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:39.928244 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eac8a06-b0a5-4c43-a3ca-78289b7d08a7 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:39.934365 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa4d346-2c91-4652-97a5-327a0fedb78b None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:39.968454 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c5f4c-adfe-44b5-9d19-624701b6984d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:39.969139 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c5f4c-adfe-44b5-9d19-624701b6984d None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:39.969427 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c5f4c-adfe-44b5-9d19-624701b6984d None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:39.969702 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c5f4c-adfe-44b5-9d19-624701b6984d None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:39.970572 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845c5f4c-adfe-44b5-9d19-624701b6984d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:39.970921 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845c5f4c-adfe-44b5-9d19-624701b6984d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:39.971667 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631d00a5-2b11-4ea6-bc69-2f05535facd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:39.972370 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845c5f4c-adfe-44b5-9d19-624701b6984d None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:39.972533 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631d00a5-2b11-4ea6-bc69-2f05535facd6 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:39.972908 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631d00a5-2b11-4ea6-bc69-2f05535facd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:39.973306 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631d00a5-2b11-4ea6-bc69-2f05535facd6 None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:39.974155 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631d00a5-2b11-4ea6-bc69-2f05535facd6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:39.974551 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631d00a5-2b11-4ea6-bc69-2f05535facd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:39.976038 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631d00a5-2b11-4ea6-bc69-2f05535facd6 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:39.989556 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 293/580] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 18 16:24:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:39.989556 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631d00a5-2b11-4ea6-bc69-2f05535facd6 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:39.999029 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 288/581] 10.222.0.26 () {56 vars in 1137 bytes} [Mon Jul 18 16:24:39 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:39.999029 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c5f4c-adfe-44b5-9d19-624701b6984d None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:40.057130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:40.096095 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631d00a5-2b11-4ea6-bc69-2f05535facd6 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:40.101140 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c5f4c-adfe-44b5-9d19-624701b6984d None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:40.120899 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:40.131241 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30239410-b630-40ff-98f3-3d8ee27fe5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:40.131952 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30239410-b630-40ff-98f3-3d8ee27fe5c5 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:40.132230 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30239410-b630-40ff-98f3-3d8ee27fe5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:40.132507 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30239410-b630-40ff-98f3-3d8ee27fe5c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:40.133241 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30239410-b630-40ff-98f3-3d8ee27fe5c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:40.133573 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30239410-b630-40ff-98f3-3d8ee27fe5c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:40.134929 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30239410-b630-40ff-98f3-3d8ee27fe5c5 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:40.138171 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7ecfd9-f6d5-435b-8d1d-5a850a630166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:40.138859 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ecfd9-f6d5-435b-8d1d-5a850a630166 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:40.139144 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ecfd9-f6d5-435b-8d1d-5a850a630166 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:40.139421 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ecfd9-f6d5-435b-8d1d-5a850a630166 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:40.140167 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7ecfd9-f6d5-435b-8d1d-5a850a630166 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:40.140477 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7ecfd9-f6d5-435b-8d1d-5a850a630166 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:40.141828 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7ecfd9-f6d5-435b-8d1d-5a850a630166 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:40.151526 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 294/582] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:40.151526 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30239410-b630-40ff-98f3-3d8ee27fe5c5 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:40.161801 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 289/583] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:40.161801 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7ecfd9-f6d5-435b-8d1d-5a850a630166 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:40.264299 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30239410-b630-40ff-98f3-3d8ee27fe5c5 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:40.270865 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7ecfd9-f6d5-435b-8d1d-5a850a630166 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:40.285722 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:40.302010 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e5e540-7a42-4e87-865d-b4cd21fce7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:40.302804 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e5e540-7a42-4e87-865d-b4cd21fce7c0 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:40.303084 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e5e540-7a42-4e87-865d-b4cd21fce7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:40.303357 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e5e540-7a42-4e87-865d-b4cd21fce7c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:40.307341 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584ecf82-3a7f-4f3d-89d6-ed0b7f473c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:40.308632 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ecf82-3a7f-4f3d-89d6-ed0b7f473c2a None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:40.309429 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ecf82-3a7f-4f3d-89d6-ed0b7f473c2a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:40.309429 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ecf82-3a7f-4f3d-89d6-ed0b7f473c2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:40.314578 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e5e540-7a42-4e87-865d-b4cd21fce7c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:40.316180 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e5e540-7a42-4e87-865d-b4cd21fce7c0 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:40.320187 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584ecf82-3a7f-4f3d-89d6-ed0b7f473c2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:40.321626 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584ecf82-3a7f-4f3d-89d6-ed0b7f473c2a None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:40.333998 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-31e5e540-7a42-4e87-865d-b4cd21fce7c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:40.333998 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:40.333998 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:40.333998 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:40.333998 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:40.333998 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:40.333998 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:40.333998 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:40.333998 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:40.333998 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:40.333998 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:40.333998 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:40.333998 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:40.333998 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:40.333998 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:40.335164 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:40.335164 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:40.335164 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:40.335164 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:40.335164 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:40.335164 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:40.335164 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:40.335164 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:40.335164 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:40.335164 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:40.335164 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:40.335164 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:40.335164 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:40.335164 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:40.335164 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:40.335164 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:40.335164 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:40.336141 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:40.336141 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:40.336141 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:40.336141 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:40.340717 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 295/584] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:40.341389 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-584ecf82-3a7f-4f3d-89d6-ed0b7f473c2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:40.341389 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:40.341389 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:40.341389 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:40.341389 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:40.341389 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:40.341389 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:40.341389 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:40.341389 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:40.341389 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:40.341389 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:40.341389 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:40.341389 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:40.341389 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:40.341389 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:40.342407 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:40.342407 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:40.342407 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:40.342407 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:40.342407 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:40.342407 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:40.342407 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:40.342407 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:40.342407 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:40.342407 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:40.342407 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:40.342407 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:40.342407 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:40.342407 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:40.342407 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:40.342407 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:40.342407 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:40.343538 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:40.343538 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:40.343538 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:40.343538 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:40.343538 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 290/585] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:40.343886 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e5e540-7a42-4e87-865d-b4cd21fce7c0 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:40.343970 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584ecf82-3a7f-4f3d-89d6-ed0b7f473c2a None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:40.381870 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:40.449965 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e5e540-7a42-4e87-865d-b4cd21fce7c0 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:40.461445 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584ecf82-3a7f-4f3d-89d6-ed0b7f473c2a None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:40.488632 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb292a1-c537-417f-8859-91e606ddbcdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:40.489451 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb292a1-c537-417f-8859-91e606ddbcdb None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:40.489720 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb292a1-c537-417f-8859-91e606ddbcdb None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:40.490001 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb292a1-c537-417f-8859-91e606ddbcdb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:40.496849 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb292a1-c537-417f-8859-91e606ddbcdb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:40.498287 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb292a1-c537-417f-8859-91e606ddbcdb None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:40.498715 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ecd76-3f08-44ac-8cbd-c96f1b37a853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:40.499763 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ecd76-3f08-44ac-8cbd-c96f1b37a853 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:40.500266 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ecd76-3f08-44ac-8cbd-c96f1b37a853 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:40.500802 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ecd76-3f08-44ac-8cbd-c96f1b37a853 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:40.508437 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463ecd76-3f08-44ac-8cbd-c96f1b37a853 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:40.509878 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463ecd76-3f08-44ac-8cbd-c96f1b37a853 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:40.512223 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-ffb292a1-c537-417f-8859-91e606ddbcdb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:40.512223 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:40.512223 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:40.512223 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:40.512223 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:40.512223 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:40.512223 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:40.512223 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:40.512223 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:40.512223 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:40.512223 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:40.512223 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:40.512223 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:24:40.512223 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:24:40.512223 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:24:40.513136 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:24:40.513136 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:40.513136 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:40.513136 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:40.513136 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:40.513136 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:40.513136 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:40.513136 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:40.513136 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:40.513136 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:40.513136 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:40.513136 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:40.513136 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:40.513136 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:40.513136 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:40.513136 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:40.513136 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:40.514018 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:24:40.514018 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:24:40.514018 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:24:40.514018 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:24:40.514018 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:24:40.514018 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:24:40.514018 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:40.514018 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:40.514018 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 296/586] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:24:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:40.514538 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb292a1-c537-417f-8859-91e606ddbcdb None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:40.530382 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-463ecd76-3f08-44ac-8cbd-c96f1b37a853 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 16:24:40.530382 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:40.530382 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:40.530382 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:40.530382 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:40.530382 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:40.530382 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:40.530382 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:40.530382 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:40.530382 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:40.530382 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:40.530382 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:40.530382 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:24:40.530382 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:24:40.530382 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:24:40.530382 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:24:40.531404 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:40.531404 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:40.531404 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:24:40.531404 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:40.531404 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:24:40.531404 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:40.531404 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:40.531404 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:40.531404 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:40.531404 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:40.531404 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:40.531404 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:40.531404 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:40.531404 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:40.531404 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:40.531404 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:40.531404 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:40.531404 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:40.532441 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:40.532441 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:40.532441 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:24:40.532441 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:24:40.532441 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:24:40.532441 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:24:40.532441 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:24:40.532441 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:24:40.532441 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 16:24:40.532441 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:40.532441 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 291/587] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:24:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:40.532441 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ecd76-3f08-44ac-8cbd-c96f1b37a853 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:40.562906 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:40.622633 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:40.628127 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb292a1-c537-417f-8859-91e606ddbcdb None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:40.645807 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ecd76-3f08-44ac-8cbd-c96f1b37a853 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:40.665841 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba69c3a-b6bd-4d8c-bcf8-1fc4bcd72501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:40.666509 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba69c3a-b6bd-4d8c-bcf8-1fc4bcd72501 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:40.666791 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba69c3a-b6bd-4d8c-bcf8-1fc4bcd72501 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:40.667063 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba69c3a-b6bd-4d8c-bcf8-1fc4bcd72501 None admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:40.669264 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba69c3a-b6bd-4d8c-bcf8-1fc4bcd72501 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:40.669576 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba69c3a-b6bd-4d8c-bcf8-1fc4bcd72501 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:40.671216 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba69c3a-b6bd-4d8c-bcf8-1fc4bcd72501 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:40.684998 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 297/588] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:40.685945 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba69c3a-b6bd-4d8c-bcf8-1fc4bcd72501 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:40.692821 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8796b7-a600-488b-aa36-8b25dd23ae4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:40.693499 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8796b7-a600-488b-aa36-8b25dd23ae4f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:40.693792 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8796b7-a600-488b-aa36-8b25dd23ae4f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:40.694075 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8796b7-a600-488b-aa36-8b25dd23ae4f None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:40.694841 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8796b7-a600-488b-aa36-8b25dd23ae4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:40.695157 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8796b7-a600-488b-aa36-8b25dd23ae4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:40.696611 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8796b7-a600-488b-aa36-8b25dd23ae4f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:40.715525 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 292/589] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:40.715525 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8796b7-a600-488b-aa36-8b25dd23ae4f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:40.805251 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba69c3a-b6bd-4d8c-bcf8-1fc4bcd72501 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:40.808558 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:40.826641 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8796b7-a600-488b-aa36-8b25dd23ae4f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:40.848488 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e326b348-7dd1-4d64-9699-02d08699ea18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:40.849153 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326b348-7dd1-4d64-9699-02d08699ea18 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:40.849428 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326b348-7dd1-4d64-9699-02d08699ea18 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:40.849701 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326b348-7dd1-4d64-9699-02d08699ea18 None admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:40.850477 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e326b348-7dd1-4d64-9699-02d08699ea18 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:40.850777 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e326b348-7dd1-4d64-9699-02d08699ea18 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:40.852225 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e326b348-7dd1-4d64-9699-02d08699ea18 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:40.864167 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 298/590] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 18 16:24:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:40.864982 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e326b348-7dd1-4d64-9699-02d08699ea18 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:40.869785 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9afe13f-6319-438d-876a-f9c07679a99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:40.870570 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afe13f-6319-438d-876a-f9c07679a99a None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:40.870860 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afe13f-6319-438d-876a-f9c07679a99a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:40.871186 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afe13f-6319-438d-876a-f9c07679a99a None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:40.872003 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9afe13f-6319-438d-876a-f9c07679a99a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:40.872314 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9afe13f-6319-438d-876a-f9c07679a99a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:40.873742 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9afe13f-6319-438d-876a-f9c07679a99a None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:40.885488 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:40.892071 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 293/591] 10.222.0.26 () {56 vars in 1137 bytes} [Mon Jul 18 16:24:40 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:40.892903 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9afe13f-6319-438d-876a-f9c07679a99a None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:40.967624 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e326b348-7dd1-4d64-9699-02d08699ea18 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:40.997201 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9afe13f-6319-438d-876a-f9c07679a99a None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:41.002515 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d80c8da-761e-42aa-af75-66d3220eb6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:41.003319 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d80c8da-761e-42aa-af75-66d3220eb6d5 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:41.003630 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d80c8da-761e-42aa-af75-66d3220eb6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:41.003908 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d80c8da-761e-42aa-af75-66d3220eb6d5 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:41.011833 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d80c8da-761e-42aa-af75-66d3220eb6d5 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:41.013298 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d80c8da-761e-42aa-af75-66d3220eb6d5 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:41.027888 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-2d80c8da-761e-42aa-af75-66d3220eb6d5 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 16:24:41.027888 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:41.027888 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:41.027888 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:41.027888 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:41.027888 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:41.027888 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:41.027888 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:41.027888 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:41.027888 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:41.027888 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:41.027888 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:41.027888 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:24:41.027888 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:24:41.027888 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:24:41.027888 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:24:41.029744 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:41.029744 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:41.029744 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:24:41.029744 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:41.029744 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:24:41.029744 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:41.029744 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:41.029744 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:41.029744 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:41.029744 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:41.029744 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:41.029744 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:41.029744 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:41.029744 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:41.029744 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:41.029744 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:41.029744 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:41.029744 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:41.031774 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:41.031774 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:41.031774 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:24:41.031774 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:24:41.031774 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:24:41.031774 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:24:41.031774 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:24:41.031774 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:24:41.031774 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 16:24:41.031774 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:41.031774 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 299/592] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 18 16:24:40 2022] GET /identity/v3/users/neutron => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:41.031774 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd0fe40-231d-4e91-8cbe-64da08800bc0 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:41.031774 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd0fe40-231d-4e91-8cbe-64da08800bc0 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:41.034760 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd0fe40-231d-4e91-8cbe-64da08800bc0 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:41.034760 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 300/593] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:41.034760 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd0fe40-231d-4e91-8cbe-64da08800bc0 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:41.038697 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8865dc-662e-4d02-afa8-b9d049665141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:41.039671 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8865dc-662e-4d02-afa8-b9d049665141 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:41.040048 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8865dc-662e-4d02-afa8-b9d049665141 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:41.040424 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8865dc-662e-4d02-afa8-b9d049665141 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:41.048148 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8865dc-662e-4d02-afa8-b9d049665141 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:41.049606 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8865dc-662e-4d02-afa8-b9d049665141 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:41.063687 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-fa8865dc-662e-4d02-afa8-b9d049665141 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:41.063687 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:41.063687 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:41.063687 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:41.063687 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:41.063687 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:41.063687 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:41.063687 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:41.063687 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:41.063687 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:41.063687 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:41.063687 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:41.063687 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:24:41.063687 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:24:41.063687 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:24:41.064755 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:24:41.064755 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:41.064755 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:41.064755 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:41.064755 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:41.064755 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:41.064755 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:41.064755 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:41.064755 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:41.064755 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:41.064755 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:41.064755 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:41.064755 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:41.064755 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:41.064755 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:41.064755 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:41.064755 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:41.070423 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:24:41.070423 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:24:41.070423 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:24:41.070423 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:24:41.070423 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:24:41.070423 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:24:41.070423 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:41.070423 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:41.070423 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 294/594] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:24:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:41.070423 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8865dc-662e-4d02-afa8-b9d049665141 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:41.070423 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:41.116429 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:41.140587 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd0fe40-231d-4e91-8cbe-64da08800bc0 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:41.169916 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8865dc-662e-4d02-afa8-b9d049665141 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:41.174976 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fe33a5-eb45-42cf-b794-34a1ea0ef6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:41.175894 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe33a5-eb45-42cf-b794-34a1ea0ef6ef None admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:41.176180 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe33a5-eb45-42cf-b794-34a1ea0ef6ef None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:41.176459 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe33a5-eb45-42cf-b794-34a1ea0ef6ef None admin] PATH_INFO: `/v3/projects/eb9747e2299c4e0991fd7edb4462c7e5/users/5b868bfa44b94ca5b470166d39f20e54/roles/4fe970e98f394bbfb096c4879bebdf97` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:41.191282 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fe33a5-eb45-42cf-b794-34a1ea0ef6ef None admin] RBAC: Authorizing `identity:create_grant(project_id=eb9747e2299c4e0991fd7edb4462c7e5, user_id=5b868bfa44b94ca5b470166d39f20e54, role_id=4fe970e98f394bbfb096c4879bebdf97)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:41.192779 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fe33a5-eb45-42cf-b794-34a1ea0ef6ef None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:41.204121 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516c8305-6808-4018-8510-436d82d716e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:41.205094 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c8305-6808-4018-8510-436d82d716e9 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:41.205238 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c8305-6808-4018-8510-436d82d716e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:41.205519 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c8305-6808-4018-8510-436d82d716e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:41.216153 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516c8305-6808-4018-8510-436d82d716e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:41.217605 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516c8305-6808-4018-8510-436d82d716e9 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:41.234308 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-516c8305-6808-4018-8510-436d82d716e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:41.234308 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:41.234308 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:41.234308 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:41.234308 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:41.234308 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:41.234308 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:41.234308 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:41.234308 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:41.234308 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:41.234308 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:41.234308 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:41.234308 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:41.234308 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:41.234308 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:41.235331 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:41.235331 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:41.235331 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:41.235331 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:41.235331 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:41.235331 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:41.235331 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:41.235331 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:41.235331 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:41.235331 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:41.235331 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:41.235331 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:41.235331 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:41.235331 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:41.235331 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:41.235331 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:41.235331 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:41.236461 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:41.236461 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:41.236461 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:41.236461 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:41.236461 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 295/595] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:41.236770 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516c8305-6808-4018-8510-436d82d716e9 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:41.286103 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 301/596] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 18 16:24:41 2022] PUT /identity/v3/projects/eb9747e2299c4e0991fd7edb4462c7e5/users/5b868bfa44b94ca5b470166d39f20e54/roles/4fe970e98f394bbfb096c4879bebdf97 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:24:41.287378 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa47e0-019d-4998-85a4-d60ad4d3afa7 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:41.287728 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa47e0-019d-4998-85a4-d60ad4d3afa7 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:41.288003 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa47e0-019d-4998-85a4-d60ad4d3afa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:41.294286 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:41.314391 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-34aa47e0-019d-4998-85a4-d60ad4d3afa7 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:41.319930 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-34aa47e0-019d-4998-85a4-d60ad4d3afa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:41.350651 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516c8305-6808-4018-8510-436d82d716e9 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:41.391371 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e37ba5f-aa4b-443c-80ad-6d6e85f111f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:41.392425 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e37ba5f-aa4b-443c-80ad-6d6e85f111f5 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:41.392959 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e37ba5f-aa4b-443c-80ad-6d6e85f111f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:41.393387 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e37ba5f-aa4b-443c-80ad-6d6e85f111f5 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:41.394468 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e37ba5f-aa4b-443c-80ad-6d6e85f111f5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:41.394928 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e37ba5f-aa4b-443c-80ad-6d6e85f111f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:41.395045 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 302/597] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:41.395891 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aa47e0-019d-4998-85a4-d60ad4d3afa7 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:41.396869 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e37ba5f-aa4b-443c-80ad-6d6e85f111f5 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:41.432425 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 296/598] 10.222.0.26 () {56 vars in 1139 bytes} [Mon Jul 18 16:24:41 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:41.433098 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e37ba5f-aa4b-443c-80ad-6d6e85f111f5 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:41.472456 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:41.532105 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aa47e0-019d-4998-85a4-d60ad4d3afa7 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:41.542081 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e37ba5f-aa4b-443c-80ad-6d6e85f111f5 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:41.558736 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:41.570157 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9cb25-338b-48a8-9d63-193cf38896a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:41.570829 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9cb25-338b-48a8-9d63-193cf38896a2 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:41.571137 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9cb25-338b-48a8-9d63-193cf38896a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:41.571387 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9cb25-338b-48a8-9d63-193cf38896a2 None admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:41.572215 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b9cb25-338b-48a8-9d63-193cf38896a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:41.572523 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b9cb25-338b-48a8-9d63-193cf38896a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:41.575139 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b9cb25-338b-48a8-9d63-193cf38896a2 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:41.579840 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c912d-6671-4c9d-bae0-f438fccc4bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:41.580503 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c912d-6671-4c9d-bae0-f438fccc4bc5 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:41.580782 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c912d-6671-4c9d-bae0-f438fccc4bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:41.581056 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c912d-6671-4c9d-bae0-f438fccc4bc5 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:41.581761 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576c912d-6671-4c9d-bae0-f438fccc4bc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:41.582070 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576c912d-6671-4c9d-bae0-f438fccc4bc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:41.583580 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576c912d-6671-4c9d-bae0-f438fccc4bc5 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:41.587519 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 303/599] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 18 16:24:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:41.588543 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9cb25-338b-48a8-9d63-193cf38896a2 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:41.600833 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 297/600] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:41.601794 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c912d-6671-4c9d-bae0-f438fccc4bc5 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:41.707786 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9cb25-338b-48a8-9d63-193cf38896a2 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:41.715962 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c912d-6671-4c9d-bae0-f438fccc4bc5 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:41.724873 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:41.743889 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c4691-fb84-4b04-9f8b-9de6989e55a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:41.744689 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c4691-fb84-4b04-9f8b-9de6989e55a1 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:41.744970 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c4691-fb84-4b04-9f8b-9de6989e55a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:41.745247 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c4691-fb84-4b04-9f8b-9de6989e55a1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:41.755414 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33617b5-6cd3-4aae-817a-d419af4e5f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:41.756237 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33617b5-6cd3-4aae-817a-d419af4e5f23 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:41.756237 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33617b5-6cd3-4aae-817a-d419af4e5f23 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:41.756237 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33617b5-6cd3-4aae-817a-d419af4e5f23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:41.757774 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80c4691-fb84-4b04-9f8b-9de6989e55a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:41.767369 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-c80c4691-fb84-4b04-9f8b-9de6989e55a1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:41.767369 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:41.767369 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:41.767369 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:41.767369 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:41.767369 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:41.767369 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:41.767369 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:41.767369 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:41.767369 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:41.767369 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:41.767369 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:41.767369 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:41.767369 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:41.767369 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:41.767369 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:41.768554 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:41.768554 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:41.768554 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:41.768554 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:41.768554 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:41.768554 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:41.768554 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:41.768554 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:41.768554 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:41.768554 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:41.768554 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:41.768554 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:41.768554 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:41.768554 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:41.768554 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:41.768554 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:41.769575 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:41.769575 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:41.769575 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:41.769575 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:41.769575 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:41.769575 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:41.769575 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:41.769575 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:41.769575 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:41.769575 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:41.769575 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:41.769575 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:41.770366 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33617b5-6cd3-4aae-817a-d419af4e5f23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:41.770450 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33617b5-6cd3-4aae-817a-d419af4e5f23 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:41.775102 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 304/601] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 18 16:24:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:41.775102 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c4691-fb84-4b04-9f8b-9de6989e55a1 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:41.791057 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-d33617b5-6cd3-4aae-817a-d419af4e5f23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:41.791057 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:41.791057 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:41.791057 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:41.791057 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:41.791057 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:41.791057 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:41.791057 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:41.791057 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:41.791057 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:41.791057 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:41.791057 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:41.791057 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:41.791057 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:41.791057 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:41.792229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:41.792229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:41.792229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:41.792229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:41.792229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:41.792229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:41.792229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:41.792229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:41.792229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:41.792229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:41.792229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:41.792229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:41.792229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:41.792229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:41.792229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:41.792229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:41.792229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:41.794190 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:41.794190 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:41.794190 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:41.794190 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:41.794190 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 298/602] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:41.794190 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33617b5-6cd3-4aae-817a-d419af4e5f23 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:41.823446 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:41.899952 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33617b5-6cd3-4aae-817a-d419af4e5f23 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:41.910970 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c4691-fb84-4b04-9f8b-9de6989e55a1 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:41.941617 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760ac2d0-86c9-4748-9c03-8595842a4e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:41.942403 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ac2d0-86c9-4748-9c03-8595842a4e30 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:41.942687 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ac2d0-86c9-4748-9c03-8595842a4e30 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:41.942959 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ac2d0-86c9-4748-9c03-8595842a4e30 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:41.950786 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760ac2d0-86c9-4748-9c03-8595842a4e30 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:41.952259 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760ac2d0-86c9-4748-9c03-8595842a4e30 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:41.956249 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad27e87a-6bb9-44c7-8271-f5ba8c8e97fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:41.957386 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27e87a-6bb9-44c7-8271-f5ba8c8e97fe None admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:41.957904 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27e87a-6bb9-44c7-8271-f5ba8c8e97fe None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:41.958383 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27e87a-6bb9-44c7-8271-f5ba8c8e97fe None admin] PATH_INFO: `/v3/projects/eb9747e2299c4e0991fd7edb4462c7e5/users/0887908bcb0d4072bf6acb4cbaf1af83/roles/4fe970e98f394bbfb096c4879bebdf97` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:41.966311 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-760ac2d0-86c9-4748-9c03-8595842a4e30 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:41.966311 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:41.966311 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:41.966311 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:41.966311 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:41.966311 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:41.966311 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:41.966311 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:41.966311 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:41.966311 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:41.966311 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:41.966311 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:41.966311 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:24:41.966311 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:24:41.966311 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:24:41.967235 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:24:41.967235 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:41.967235 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:41.967235 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:41.967235 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:41.967235 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:41.967235 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:41.967235 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:41.967235 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:41.967235 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:41.967235 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:41.967235 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:41.967235 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:41.967235 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:41.967235 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:41.967235 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:41.967235 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:41.968178 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:24:41.968178 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:24:41.968178 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:24:41.968178 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:24:41.968178 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:24:41.968178 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:24:41.968178 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:41.968178 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:41.968178 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 299/603] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:24:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:41.968674 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760ac2d0-86c9-4748-9c03-8595842a4e30 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:41.982889 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad27e87a-6bb9-44c7-8271-f5ba8c8e97fe None admin] RBAC: Authorizing `identity:create_grant(project_id=eb9747e2299c4e0991fd7edb4462c7e5, user_id=0887908bcb0d4072bf6acb4cbaf1af83, role_id=4fe970e98f394bbfb096c4879bebdf97)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:41.984754 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad27e87a-6bb9-44c7-8271-f5ba8c8e97fe None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:42.059849 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 305/604] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 18 16:24:41 2022] PUT /identity/v3/projects/eb9747e2299c4e0991fd7edb4462c7e5/users/0887908bcb0d4072bf6acb4cbaf1af83/roles/4fe970e98f394bbfb096c4879bebdf97 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:24:42.060659 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:42.062223 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad27e87a-6bb9-44c7-8271-f5ba8c8e97fe None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:42.084464 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760ac2d0-86c9-4748-9c03-8595842a4e30 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:42.120066 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:42.140793 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052c6e33-d7a2-434b-b315-f419a56fc284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:42.142185 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c6e33-d7a2-434b-b315-f419a56fc284 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:42.143092 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c6e33-d7a2-434b-b315-f419a56fc284 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:42.143607 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c6e33-d7a2-434b-b315-f419a56fc284 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:42.144823 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052c6e33-d7a2-434b-b315-f419a56fc284 None admin] RBAC: Adding query filter params (name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:42.145294 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052c6e33-d7a2-434b-b315-f419a56fc284 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:42.148003 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052c6e33-d7a2-434b-b315-f419a56fc284 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:42.164478 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 300/605] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 18 16:24:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:42.165674 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052c6e33-d7a2-434b-b315-f419a56fc284 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:42.227835 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad27e87a-6bb9-44c7-8271-f5ba8c8e97fe None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:42.278025 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90eeb8d-d5e5-4f2d-91d3-c65f1ce1013e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:42.278820 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90eeb8d-d5e5-4f2d-91d3-c65f1ce1013e None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:42.279180 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90eeb8d-d5e5-4f2d-91d3-c65f1ce1013e None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:42.279551 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90eeb8d-d5e5-4f2d-91d3-c65f1ce1013e None admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:42.280355 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90eeb8d-d5e5-4f2d-91d3-c65f1ce1013e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:42.280728 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90eeb8d-d5e5-4f2d-91d3-c65f1ce1013e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:42.282181 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90eeb8d-d5e5-4f2d-91d3-c65f1ce1013e None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:42.289199 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052c6e33-d7a2-434b-b315-f419a56fc284 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:42.295054 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 306/606] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:42.296055 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90eeb8d-d5e5-4f2d-91d3-c65f1ce1013e None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:42.345240 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae5562-6ce4-4a00-a53e-10ff7fa265d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:42.346073 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae5562-6ce4-4a00-a53e-10ff7fa265d7 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:42.346460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae5562-6ce4-4a00-a53e-10ff7fa265d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:42.346835 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae5562-6ce4-4a00-a53e-10ff7fa265d7 None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:42.347768 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eae5562-6ce4-4a00-a53e-10ff7fa265d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:42.348192 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eae5562-6ce4-4a00-a53e-10ff7fa265d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:42.349790 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eae5562-6ce4-4a00-a53e-10ff7fa265d7 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:42.364052 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 301/607] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 18 16:24:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:42.365042 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae5562-6ce4-4a00-a53e-10ff7fa265d7 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:42.377150 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:42.408349 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:42.430297 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90eeb8d-d5e5-4f2d-91d3-c65f1ce1013e None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:42.473143 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b842c3-6f3a-43a4-af04-efd1b103156a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:42.474170 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b842c3-6f3a-43a4-af04-efd1b103156a None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:42.474553 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b842c3-6f3a-43a4-af04-efd1b103156a None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:42.474931 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b842c3-6f3a-43a4-af04-efd1b103156a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:42.484396 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae5562-6ce4-4a00-a53e-10ff7fa265d7 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:42.487336 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b842c3-6f3a-43a4-af04-efd1b103156a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:42.488987 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b842c3-6f3a-43a4-af04-efd1b103156a None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:42.507461 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-48b842c3-6f3a-43a4-af04-efd1b103156a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:42.507461 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:42.507461 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:42.507461 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:42.507461 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:42.507461 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:42.507461 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:42.507461 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:42.507461 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:42.507461 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:42.507461 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:42.507461 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:42.507461 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:42.507461 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:42.507461 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:42.508596 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:42.508596 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:42.508596 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:42.508596 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:42.508596 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:42.508596 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:42.508596 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:42.508596 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:42.508596 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:42.508596 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:42.508596 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:42.508596 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:42.508596 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:42.508596 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:42.508596 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:42.508596 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:42.508596 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:42.509657 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:42.509657 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:42.509657 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:42.509657 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:42.511871 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 307/608] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:42.512731 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b842c3-6f3a-43a4-af04-efd1b103156a None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:42.528250 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263869a4-a726-4eee-9775-476a4bd51fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:42.529403 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263869a4-a726-4eee-9775-476a4bd51fff None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:42.530198 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263869a4-a726-4eee-9775-476a4bd51fff None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:42.530779 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263869a4-a726-4eee-9775-476a4bd51fff None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:42.539426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263869a4-a726-4eee-9775-476a4bd51fff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:42.541742 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263869a4-a726-4eee-9775-476a4bd51fff None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:42.564758 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-263869a4-a726-4eee-9775-476a4bd51fff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:42.564758 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:42.564758 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:42.564758 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:42.564758 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:42.564758 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:42.564758 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:42.564758 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:42.564758 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:42.564758 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:42.564758 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:42.564758 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:42.564758 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:24:42.564758 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:24:42.564758 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:24:42.565894 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:24:42.565894 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:42.565894 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:42.565894 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:42.565894 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:42.565894 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:42.565894 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:42.565894 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:42.565894 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:42.565894 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:42.565894 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:42.565894 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:42.565894 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:42.565894 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:42.565894 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:42.565894 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:42.565894 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:42.566948 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:24:42.566948 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:24:42.566948 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:24:42.566948 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:24:42.566948 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:24:42.566948 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:24:42.566948 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:42.566948 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:42.575467 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 302/609] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:24:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:42.576582 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263869a4-a726-4eee-9775-476a4bd51fff None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:42.631211 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b842c3-6f3a-43a4-af04-efd1b103156a None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:42.670937 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:42.677029 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfabbda3-38f4-4040-8f5a-cb952ab7f5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:42.678069 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfabbda3-38f4-4040-8f5a-cb952ab7f5c6 None admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:42.678475 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfabbda3-38f4-4040-8f5a-cb952ab7f5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:42.678849 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfabbda3-38f4-4040-8f5a-cb952ab7f5c6 None admin] PATH_INFO: `/v3/projects/eb9747e2299c4e0991fd7edb4462c7e5/users/6f2e55df807c43d7951fa4be67ef5e81/roles/4fe970e98f394bbfb096c4879bebdf97` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:42.691939 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:42.699836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfabbda3-38f4-4040-8f5a-cb952ab7f5c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb9747e2299c4e0991fd7edb4462c7e5, user_id=6f2e55df807c43d7951fa4be67ef5e81, role_id=4fe970e98f394bbfb096c4879bebdf97)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:42.701468 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfabbda3-38f4-4040-8f5a-cb952ab7f5c6 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:42.712902 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263869a4-a726-4eee-9775-476a4bd51fff None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:42.752337 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0239ff-8fcc-44e9-a400-2e9481073687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:42.753158 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0239ff-8fcc-44e9-a400-2e9481073687 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:42.753562 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0239ff-8fcc-44e9-a400-2e9481073687 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:42.753993 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0239ff-8fcc-44e9-a400-2e9481073687 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:42.754867 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0239ff-8fcc-44e9-a400-2e9481073687 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:42.755272 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0239ff-8fcc-44e9-a400-2e9481073687 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:42.756794 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0239ff-8fcc-44e9-a400-2e9481073687 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:42.770352 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 303/610] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:42.771199 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0239ff-8fcc-44e9-a400-2e9481073687 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:42.797738 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 308/611] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 18 16:24:42 2022] PUT /identity/v3/projects/eb9747e2299c4e0991fd7edb4462c7e5/users/6f2e55df807c43d7951fa4be67ef5e81/roles/4fe970e98f394bbfb096c4879bebdf97 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:24:42.805081 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfabbda3-38f4-4040-8f5a-cb952ab7f5c6 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:42.921354 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0239ff-8fcc-44e9-a400-2e9481073687 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:42.957042 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:42.968046 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a62dec0-758c-4c4a-9f5a-463881855a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:42.969124 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62dec0-758c-4c4a-9f5a-463881855a53 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:42.969722 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62dec0-758c-4c4a-9f5a-463881855a53 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:42.970166 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62dec0-758c-4c4a-9f5a-463881855a53 None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:42.971371 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a62dec0-758c-4c4a-9f5a-463881855a53 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:42.971872 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a62dec0-758c-4c4a-9f5a-463881855a53 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:42.973965 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a62dec0-758c-4c4a-9f5a-463881855a53 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:42.990306 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 304/612] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 18 16:24:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:42.997752 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a62dec0-758c-4c4a-9f5a-463881855a53 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:43.072722 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfabbda3-38f4-4040-8f5a-cb952ab7f5c6 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:43.115100 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8642a39-9770-410f-bf8f-c63785d0e75a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:43.116136 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8642a39-9770-410f-bf8f-c63785d0e75a None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:43.116517 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8642a39-9770-410f-bf8f-c63785d0e75a None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:43.116945 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8642a39-9770-410f-bf8f-c63785d0e75a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:43.127723 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8642a39-9770-410f-bf8f-c63785d0e75a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:43.129459 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8642a39-9770-410f-bf8f-c63785d0e75a None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:43.133903 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a62dec0-758c-4c4a-9f5a-463881855a53 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:43.148318 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-b8642a39-9770-410f-bf8f-c63785d0e75a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 16:24:43.148318 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:43.148318 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:43.148318 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:43.148318 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:43.148318 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:43.148318 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:43.148318 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:43.148318 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:43.148318 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:43.148318 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:43.148318 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:43.148318 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:24:43.148318 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:24:43.148318 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:24:43.148318 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:24:43.149534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:43.149534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:43.149534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:24:43.149534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:43.149534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:24:43.149534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:43.149534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:43.149534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:43.149534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:43.149534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:43.149534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:43.149534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:43.149534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:43.149534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:43.149534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:43.149534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:43.149534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:43.149534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:43.150702 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:43.150702 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:43.150702 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:24:43.150702 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:24:43.150702 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:24:43.150702 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:24:43.150702 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:24:43.150702 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:24:43.150702 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 16:24:43.150702 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:43.152937 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 309/613] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 18 16:24:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:43.160904 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8642a39-9770-410f-bf8f-c63785d0e75a None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:43.183192 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929a904f-fda4-4034-9a68-13a8fe6cbbc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:43.191517 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a904f-fda4-4034-9a68-13a8fe6cbbc3 None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:43.191517 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a904f-fda4-4034-9a68-13a8fe6cbbc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:43.191517 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a904f-fda4-4034-9a68-13a8fe6cbbc3 None admin] PATH_INFO: `/v3/projects/eb9747e2299c4e0991fd7edb4462c7e5/users/7a6054dbfbbc487794d2a32061b0ac82/roles/4fe970e98f394bbfb096c4879bebdf97` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:43.225059 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929a904f-fda4-4034-9a68-13a8fe6cbbc3 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb9747e2299c4e0991fd7edb4462c7e5, user_id=7a6054dbfbbc487794d2a32061b0ac82, role_id=4fe970e98f394bbfb096c4879bebdf97)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:43.226562 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929a904f-fda4-4034-9a68-13a8fe6cbbc3 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:43.250807 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:43.315188 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 305/614] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 18 16:24:42 2022] PUT /identity/v3/projects/eb9747e2299c4e0991fd7edb4462c7e5/users/7a6054dbfbbc487794d2a32061b0ac82/roles/4fe970e98f394bbfb096c4879bebdf97 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:24:43.316134 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8642a39-9770-410f-bf8f-c63785d0e75a None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:43.369296 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504a8833-d59a-4ad5-8946-f6e6cac6a8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:43.369910 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a8833-d59a-4ad5-8946-f6e6cac6a8ff None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:43.370250 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a8833-d59a-4ad5-8946-f6e6cac6a8ff None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:43.370856 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a8833-d59a-4ad5-8946-f6e6cac6a8ff None admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:43.371752 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504a8833-d59a-4ad5-8946-f6e6cac6a8ff None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:43.372089 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504a8833-d59a-4ad5-8946-f6e6cac6a8ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:43.373571 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504a8833-d59a-4ad5-8946-f6e6cac6a8ff None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:43.381031 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4db25-7474-4f16-a4f3-15127dde5e49 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:43.381358 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4db25-7474-4f16-a4f3-15127dde5e49 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:43.381600 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4db25-7474-4f16-a4f3-15127dde5e49 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:43.383322 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 306/615] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:43.389508 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74482810-12c0-49e7-8681-24a13f528083 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:43.390915 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74482810-12c0-49e7-8681-24a13f528083 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:43.390915 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74482810-12c0-49e7-8681-24a13f528083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:43.399609 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 310/616] 10.222.0.26 () {56 vars in 1135 bytes} [Mon Jul 18 16:24:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:43.406086 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504a8833-d59a-4ad5-8946-f6e6cac6a8ff None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:43.431134 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-74482810-12c0-49e7-8681-24a13f528083 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:43.432696 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-74482810-12c0-49e7-8681-24a13f528083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:43.504443 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:43.554765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 307/617] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:43.588623 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504a8833-d59a-4ad5-8946-f6e6cac6a8ff None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:43.590553 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74482810-12c0-49e7-8681-24a13f528083 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:43.611896 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:43.679343 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d106adde-eca2-44cd-8388-4e875f2ae83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:43.680462 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d106adde-eca2-44cd-8388-4e875f2ae83d None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:43.680924 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d106adde-eca2-44cd-8388-4e875f2ae83d None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:43.681586 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d106adde-eca2-44cd-8388-4e875f2ae83d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:43.702287 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d106adde-eca2-44cd-8388-4e875f2ae83d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:43.704833 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d106adde-eca2-44cd-8388-4e875f2ae83d None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:43.739109 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-d106adde-eca2-44cd-8388-4e875f2ae83d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:43.739109 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:43.739109 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:43.739109 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:43.739109 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:43.739109 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:43.739109 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:43.739109 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:43.739109 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:43.739109 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:43.739109 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:43.739109 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:43.739109 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:43.739109 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:43.739109 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:43.740081 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:43.740081 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:43.740081 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:43.740081 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:43.740081 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:43.740081 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:43.740081 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:43.740081 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:43.740081 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:43.740081 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:43.740081 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:43.740081 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:43.740081 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:43.740081 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:43.740081 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:43.740081 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:43.740081 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:43.741115 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:43.741115 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:43.741115 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:43.741115 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:43.752907 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 311/618] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:24:43.759429 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d106adde-eca2-44cd-8388-4e875f2ae83d None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:43.784692 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74482810-12c0-49e7-8681-24a13f528083 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:43.852123 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff90dfc-045e-4d9f-a0d0-aa03ac7493b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:43.853973 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff90dfc-045e-4d9f-a0d0-aa03ac7493b2 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:43.855996 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff90dfc-045e-4d9f-a0d0-aa03ac7493b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:43.856587 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff90dfc-045e-4d9f-a0d0-aa03ac7493b2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:43.882624 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff90dfc-045e-4d9f-a0d0-aa03ac7493b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:43.893499 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-5ff90dfc-045e-4d9f-a0d0-aa03ac7493b2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:43.893499 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:43.893499 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:43.893499 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:43.893499 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:43.893499 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:43.893499 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:43.893499 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:43.893499 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:43.893499 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:43.893499 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:43.893499 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:43.893499 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:43.893499 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:43.893499 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:43.893499 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:43.894739 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:43.894739 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:43.894739 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:43.894739 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:43.894739 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:43.894739 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:43.894739 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:43.894739 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:43.894739 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:43.894739 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:43.894739 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:43.894739 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:43.894739 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:43.894739 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:43.894739 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:43.894739 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:43.895810 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:43.895810 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:43.895810 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:43.895810 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:43.895810 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:43.895810 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:43.895810 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:43.895810 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:43.895810 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:43.895810 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:43.895810 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:43.895810 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:43.898054 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 308/619] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 18 16:24:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:24:43.903224 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff90dfc-045e-4d9f-a0d0-aa03ac7493b2 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:43.947162 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d106adde-eca2-44cd-8388-4e875f2ae83d None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:43.980688 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:43.991793 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b4a50d-10fb-4e7d-add0-8af2b9e80e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:43.997338 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4a50d-10fb-4e7d-add0-8af2b9e80e85 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:43.997338 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4a50d-10fb-4e7d-add0-8af2b9e80e85 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:43.997338 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4a50d-10fb-4e7d-add0-8af2b9e80e85 None admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:43.997338 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b4a50d-10fb-4e7d-add0-8af2b9e80e85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:43.997338 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b4a50d-10fb-4e7d-add0-8af2b9e80e85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:43.997338 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b4a50d-10fb-4e7d-add0-8af2b9e80e85 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:44.029767 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 312/620] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:44.031835 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b4a50d-10fb-4e7d-add0-8af2b9e80e85 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:44.070042 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:44.111397 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff90dfc-045e-4d9f-a0d0-aa03ac7493b2 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:44.193593 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae60931c-ad12-48ad-8b91-d9c7dc02803a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:44.194265 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae60931c-ad12-48ad-8b91-d9c7dc02803a None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:44.194684 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae60931c-ad12-48ad-8b91-d9c7dc02803a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:44.194968 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae60931c-ad12-48ad-8b91-d9c7dc02803a None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:44.195753 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae60931c-ad12-48ad-8b91-d9c7dc02803a None admin] RBAC: Adding query filter params (name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:44.196062 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae60931c-ad12-48ad-8b91-d9c7dc02803a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:44.197488 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae60931c-ad12-48ad-8b91-d9c7dc02803a None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:44.209358 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b4a50d-10fb-4e7d-add0-8af2b9e80e85 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:44.217631 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 309/621] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 18 16:24:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:44.223160 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae60931c-ad12-48ad-8b91-d9c7dc02803a None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:44.281032 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2792d753-63e8-4f92-b973-fded506e7abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:44.281836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2792d753-63e8-4f92-b973-fded506e7abe None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:44.282117 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2792d753-63e8-4f92-b973-fded506e7abe None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:44.282392 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2792d753-63e8-4f92-b973-fded506e7abe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:44.298762 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2792d753-63e8-4f92-b973-fded506e7abe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:44.300369 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2792d753-63e8-4f92-b973-fded506e7abe None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:44.317072 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-2792d753-63e8-4f92-b973-fded506e7abe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:44.317072 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:44.317072 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:44.317072 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:44.317072 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:44.317072 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:44.317072 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:44.317072 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:44.317072 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:44.317072 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:44.317072 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:44.317072 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:44.317072 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:24:44.317072 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:24:44.317072 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:24:44.318036 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:24:44.318036 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:44.318036 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:44.318036 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:44.318036 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:44.318036 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:44.318036 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:44.318036 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:44.318036 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:44.318036 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:44.318036 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:44.318036 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:44.318036 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:44.318036 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:44.318036 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:44.318036 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:44.318036 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:44.318918 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:24:44.318918 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:24:44.318918 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:24:44.318918 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:24:44.318918 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:24:44.318918 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:24:44.318918 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:44.318918 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:44.318918 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 313/622] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:24:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:44.323566 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6bb5b-a6a8-4605-8621-cc3d0d703650 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:44.323566 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6bb5b-a6a8-4605-8621-cc3d0d703650 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:44.323566 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6bb5b-a6a8-4605-8621-cc3d0d703650 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:44.329766 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 314/623] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:44.330649 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f6bb5b-a6a8-4605-8621-cc3d0d703650 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:44.357258 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:44.369353 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae60931c-ad12-48ad-8b91-d9c7dc02803a None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:44.417586 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6898ae-6e4b-4174-9253-3d53ca78f059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:44.418608 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6898ae-6e4b-4174-9253-3d53ca78f059 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:44.418932 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6898ae-6e4b-4174-9253-3d53ca78f059 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:44.419304 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6898ae-6e4b-4174-9253-3d53ca78f059 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:44.427095 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:44.433438 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6898ae-6e4b-4174-9253-3d53ca78f059 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:44.435599 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6898ae-6e4b-4174-9253-3d53ca78f059 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:44.452706 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f6bb5b-a6a8-4605-8621-cc3d0d703650 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:44.461723 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-bb6898ae-6e4b-4174-9253-3d53ca78f059 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:44.461723 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:44.461723 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:44.461723 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:44.461723 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:44.461723 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:44.461723 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:44.461723 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:44.461723 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:44.461723 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:44.461723 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:44.461723 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:44.461723 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:44.461723 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:44.461723 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:44.462724 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:44.462724 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:44.462724 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:44.462724 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:44.462724 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:44.462724 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:44.462724 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:44.462724 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:44.462724 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:44.462724 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:44.462724 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:44.462724 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:44.462724 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:44.462724 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:44.462724 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:44.462724 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:44.462724 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:44.464056 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:44.464056 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:44.464056 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:44.464056 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:44.464056 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 310/624] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:24:44.465565 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd669c-2460-4ca1-b761-2d373ab642e9 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:44.465913 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd669c-2460-4ca1-b761-2d373ab642e9 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:44.466290 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd669c-2460-4ca1-b761-2d373ab642e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:44.491769 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-11fd669c-2460-4ca1-b761-2d373ab642e9 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:44.493260 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-11fd669c-2460-4ca1-b761-2d373ab642e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:44.495008 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd3de76-3337-429a-9c7c-909f1c3b479f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:44.496400 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3de76-3337-429a-9c7c-909f1c3b479f None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:44.496737 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3de76-3337-429a-9c7c-909f1c3b479f None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:44.497018 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3de76-3337-429a-9c7c-909f1c3b479f None admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:44.497850 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd3de76-3337-429a-9c7c-909f1c3b479f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:44.498162 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd3de76-3337-429a-9c7c-909f1c3b479f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:44.499642 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd3de76-3337-429a-9c7c-909f1c3b479f None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:44.512856 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 315/625] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 18 16:24:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:44.513697 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd3de76-3337-429a-9c7c-909f1c3b479f None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:44.585084 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 311/626] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:44.585993 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd669c-2460-4ca1-b761-2d373ab642e9 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:44.631389 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd3de76-3337-429a-9c7c-909f1c3b479f None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:44.674946 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cd77de-aa57-4605-8806-3879586d3006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:44.675659 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd77de-aa57-4605-8806-3879586d3006 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:44.675942 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd77de-aa57-4605-8806-3879586d3006 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:44.676477 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd77de-aa57-4605-8806-3879586d3006 None admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:44.677829 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cd77de-aa57-4605-8806-3879586d3006 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:44.678146 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cd77de-aa57-4605-8806-3879586d3006 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:44.679823 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cd77de-aa57-4605-8806-3879586d3006 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:44.682235 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:44.690774 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:44.697607 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 316/627] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:44.700154 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cd77de-aa57-4605-8806-3879586d3006 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:44.732351 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd669c-2460-4ca1-b761-2d373ab642e9 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:44.780178 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda93aab-3b82-4278-963d-06f1196c9ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:44.781070 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda93aab-3b82-4278-963d-06f1196c9ea8 None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:44.781351 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda93aab-3b82-4278-963d-06f1196c9ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:44.781626 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda93aab-3b82-4278-963d-06f1196c9ea8 None admin] PATH_INFO: `/v3/projects/eb9747e2299c4e0991fd7edb4462c7e5/users/43854d7a6df2466eb7976fabe6011767/roles/4fe970e98f394bbfb096c4879bebdf97` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:44.807292 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda93aab-3b82-4278-963d-06f1196c9ea8 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb9747e2299c4e0991fd7edb4462c7e5, user_id=43854d7a6df2466eb7976fabe6011767, role_id=4fe970e98f394bbfb096c4879bebdf97)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:44.810497 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda93aab-3b82-4278-963d-06f1196c9ea8 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:44.819772 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cd77de-aa57-4605-8806-3879586d3006 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:44.864653 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ed2f6f-9c70-4b63-bf70-6d08c18769b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:44.865729 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed2f6f-9c70-4b63-bf70-6d08c18769b5 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:44.866176 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed2f6f-9c70-4b63-bf70-6d08c18769b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:44.866595 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed2f6f-9c70-4b63-bf70-6d08c18769b5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:44.878259 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ed2f6f-9c70-4b63-bf70-6d08c18769b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:44.887194 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-34ed2f6f-9c70-4b63-bf70-6d08c18769b5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:44.887194 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:44.887194 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:44.887194 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:44.887194 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:44.887194 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:44.887194 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:44.887194 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:44.887194 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:44.887194 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:44.887194 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:44.887194 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:44.887194 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:44.887194 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:44.887194 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:44.887194 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:44.888454 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:44.888454 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:44.888454 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:44.888454 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:44.888454 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:44.888454 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:44.888454 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:44.888454 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:44.888454 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:44.888454 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:44.888454 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:44.888454 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:44.888454 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:44.888454 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:44.888454 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:44.888454 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:44.889620 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:44.889620 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:44.889620 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:44.889620 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:44.889620 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:44.889620 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:44.889620 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:44.889620 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:44.889620 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:44.889620 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:44.889620 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:44.889620 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:44.891679 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 312/628] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 18 16:24:44 2022] PUT /identity/v3/projects/eb9747e2299c4e0991fd7edb4462c7e5/users/43854d7a6df2466eb7976fabe6011767/roles/4fe970e98f394bbfb096c4879bebdf97 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:24:44.892262 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 317/629] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 18 16:24:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:44.896132 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda93aab-3b82-4278-963d-06f1196c9ea8 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:44.898777 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ed2f6f-9c70-4b63-bf70-6d08c18769b5 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:45.023191 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:45.089216 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda93aab-3b82-4278-963d-06f1196c9ea8 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:45.092162 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ed2f6f-9c70-4b63-bf70-6d08c18769b5 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:45.120056 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:45.132824 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c685613f-a89b-49ef-b57b-2b854519b32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:45.133601 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685613f-a89b-49ef-b57b-2b854519b32f None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:45.133985 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685613f-a89b-49ef-b57b-2b854519b32f None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:45.134354 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685613f-a89b-49ef-b57b-2b854519b32f None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:45.135210 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c685613f-a89b-49ef-b57b-2b854519b32f None admin] RBAC: Adding query filter params (name=service) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:45.135694 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c685613f-a89b-49ef-b57b-2b854519b32f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:45.136948 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e603346-9826-4e42-aeb5-d8dc8264a4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:45.137851 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c685613f-a89b-49ef-b57b-2b854519b32f None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:45.138357 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e603346-9826-4e42-aeb5-d8dc8264a4c9 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:45.138800 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e603346-9826-4e42-aeb5-d8dc8264a4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:45.139312 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e603346-9826-4e42-aeb5-d8dc8264a4c9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:45.149683 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e603346-9826-4e42-aeb5-d8dc8264a4c9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:45.150017 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 318/630] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 18 16:24:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:45.151576 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e603346-9826-4e42-aeb5-d8dc8264a4c9 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:45.158167 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c685613f-a89b-49ef-b57b-2b854519b32f None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:45.170437 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-6e603346-9826-4e42-aeb5-d8dc8264a4c9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:24:45.170437 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:45.170437 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:45.170437 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:45.170437 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:45.170437 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:45.170437 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:45.170437 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:45.170437 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:45.170437 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:45.170437 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:45.170437 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:45.170437 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:24:45.170437 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:24:45.170437 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:24:45.170437 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:24:45.172991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:45.172991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:45.172991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:24:45.172991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:45.172991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:24:45.172991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:45.172991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:45.172991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:45.172991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:45.172991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:45.172991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:45.172991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:45.172991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:45.172991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:45.172991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:45.172991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:45.172991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:45.172991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:45.173925 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:45.173925 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:45.173925 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:24:45.173925 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:24:45.173925 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:24:45.173925 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:24:45.173925 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:24:45.173925 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:24:45.173925 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:24:45.173925 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:45.175166 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 313/631] 10.222.0.26 () {56 vars in 1093 bytes} [Mon Jul 18 16:24:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:45.180375 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e603346-9826-4e42-aeb5-d8dc8264a4c9 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:45.322863 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c685613f-a89b-49ef-b57b-2b854519b32f None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:45.347756 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:45.369342 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0692f867-1c90-4701-a10c-9b56722d2238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:45.370573 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692f867-1c90-4701-a10c-9b56722d2238 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:45.371136 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692f867-1c90-4701-a10c-9b56722d2238 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:45.371941 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692f867-1c90-4701-a10c-9b56722d2238 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:45.390931 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0692f867-1c90-4701-a10c-9b56722d2238 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:45.392846 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0692f867-1c90-4701-a10c-9b56722d2238 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:45.415853 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-0692f867-1c90-4701-a10c-9b56722d2238 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:45.415853 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:45.415853 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:45.415853 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:45.415853 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:45.415853 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:45.415853 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:45.415853 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:45.415853 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:45.415853 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:45.415853 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:45.415853 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:45.415853 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:45.415853 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:45.415853 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:45.416812 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:45.416812 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:45.416812 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:45.416812 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:45.416812 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:45.416812 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:45.416812 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:45.416812 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:45.416812 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:45.416812 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:45.416812 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:45.416812 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:45.416812 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:45.416812 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:45.416812 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:45.416812 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:45.416812 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:45.417742 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:45.417742 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:45.417742 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:45.417742 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:45.420484 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e603346-9826-4e42-aeb5-d8dc8264a4c9 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:45.422229 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 319/632] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:45.423993 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5b785e-311f-4713-9a4e-8246e4c22bda None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:45.424287 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5b785e-311f-4713-9a4e-8246e4c22bda None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:45.424564 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5b785e-311f-4713-9a4e-8246e4c22bda None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:45.426503 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 320/633] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:45.427929 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5b785e-311f-4713-9a4e-8246e4c22bda None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:45.461111 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1172aee6-52f1-41cc-a0c0-614a832183d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:45.462269 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1172aee6-52f1-41cc-a0c0-614a832183d0 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:45.462761 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1172aee6-52f1-41cc-a0c0-614a832183d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:45.463260 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1172aee6-52f1-41cc-a0c0-614a832183d0 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:45.464779 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1172aee6-52f1-41cc-a0c0-614a832183d0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:45.465313 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1172aee6-52f1-41cc-a0c0-614a832183d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:45.467118 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1172aee6-52f1-41cc-a0c0-614a832183d0 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:45.489441 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 314/634] 10.222.0.26 () {56 vars in 1133 bytes} [Mon Jul 18 16:24:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:45.491196 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee80d8c-fd74-48cf-b682-85d5fadb2a50 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:45.491758 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee80d8c-fd74-48cf-b682-85d5fadb2a50 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:45.492294 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee80d8c-fd74-48cf-b682-85d5fadb2a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:45.515986 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-aee80d8c-fd74-48cf-b682-85d5fadb2a50 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:45.517194 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-aee80d8c-fd74-48cf-b682-85d5fadb2a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:45.557687 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5b785e-311f-4713-9a4e-8246e4c22bda None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:45.565361 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:45.603643 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96210cf6-b087-41fc-b6bd-2b3569b6ec01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:45.604300 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96210cf6-b087-41fc-b6bd-2b3569b6ec01 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:45.604860 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96210cf6-b087-41fc-b6bd-2b3569b6ec01 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:45.605396 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96210cf6-b087-41fc-b6bd-2b3569b6ec01 None admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:45.606974 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96210cf6-b087-41fc-b6bd-2b3569b6ec01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:45.607356 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96210cf6-b087-41fc-b6bd-2b3569b6ec01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:45.609217 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96210cf6-b087-41fc-b6bd-2b3569b6ec01 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:45.621639 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 315/635] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:45.623255 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee80d8c-fd74-48cf-b682-85d5fadb2a50 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:45.624499 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 321/636] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:45.630245 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96210cf6-b087-41fc-b6bd-2b3569b6ec01 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:45.682451 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:45.774011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee80d8c-fd74-48cf-b682-85d5fadb2a50 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:45.782642 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96210cf6-b087-41fc-b6bd-2b3569b6ec01 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:45.812510 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d09f8-ae21-4560-b876-d10010f76fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:45.813172 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d09f8-ae21-4560-b876-d10010f76fe7 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:45.813459 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d09f8-ae21-4560-b876-d10010f76fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:45.813744 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d09f8-ae21-4560-b876-d10010f76fe7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:45.831057 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcf39e1-409a-4911-afa4-ef79abf722cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:45.832247 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf39e1-409a-4911-afa4-ef79abf722cf None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:45.832753 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf39e1-409a-4911-afa4-ef79abf722cf None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:45.833224 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf39e1-409a-4911-afa4-ef79abf722cf None admin] PATH_INFO: `/v3/users/glance` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:45.844208 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcf39e1-409a-4911-afa4-ef79abf722cf None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:45.845653 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcf39e1-409a-4911-afa4-ef79abf722cf None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:45.860865 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-2bcf39e1-409a-4911-afa4-ef79abf722cf None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 18 16:24:45.860865 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:45.860865 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:45.860865 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:45.860865 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:45.860865 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:45.860865 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:45.860865 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:45.860865 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:45.860865 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:45.860865 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:45.860865 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:45.860865 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:24:45.860865 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:24:45.860865 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:24:45.860865 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:24:45.861837 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:45.861837 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:45.861837 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:24:45.861837 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:45.861837 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:24:45.861837 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:45.861837 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:45.861837 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:45.861837 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:45.861837 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:45.861837 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:45.861837 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:45.861837 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:45.861837 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:45.861837 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:45.861837 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:45.861837 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:45.861837 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:45.862767 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:45.862767 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:45.862767 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:24:45.862767 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:24:45.862767 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:24:45.862767 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:24:45.862767 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:24:45.862767 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:24:45.862767 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 18 16:24:45.862767 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:45.866167 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:45.866546 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 322/637] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:24:45 2022] GET /identity/v3/users/glance => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:45.867515 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcf39e1-409a-4911-afa4-ef79abf722cf None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:45.895419 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d09f8-ae21-4560-b876-d10010f76fe7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:45.897055 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d09f8-ae21-4560-b876-d10010f76fe7 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:45.962120 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:45.971036 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcf39e1-409a-4911-afa4-ef79abf722cf None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:46.009610 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d227c0-11f2-4aa2-89c3-e3474a01fba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:46.010567 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d227c0-11f2-4aa2-89c3-e3474a01fba5 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:46.010976 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d227c0-11f2-4aa2-89c3-e3474a01fba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:46.011420 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d227c0-11f2-4aa2-89c3-e3474a01fba5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:46.018868 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 316/638] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:24:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:24:46.020303 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b88873-d5e2-42b0-92ba-6dec2dbcb3ff None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:46.020593 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b88873-d5e2-42b0-92ba-6dec2dbcb3ff None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:46.020870 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b88873-d5e2-42b0-92ba-6dec2dbcb3ff None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:46.026720 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 317/639] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:46.026720 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b88873-d5e2-42b0-92ba-6dec2dbcb3ff None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:46.028019 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d227c0-11f2-4aa2-89c3-e3474a01fba5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:46.037149 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-41d227c0-11f2-4aa2-89c3-e3474a01fba5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:46.037149 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:46.037149 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:46.037149 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:46.037149 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:46.037149 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:46.037149 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:46.037149 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:46.037149 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:46.037149 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:46.037149 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:46.037149 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:46.037149 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:46.037149 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:46.037149 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:46.037149 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:46.038516 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:46.038516 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:46.038516 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:46.038516 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:46.038516 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:46.038516 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:46.038516 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:46.038516 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:46.038516 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:46.038516 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:46.038516 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:46.038516 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:46.038516 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:46.038516 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:46.038516 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:46.038516 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:46.039477 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:46.039477 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:46.039477 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:46.039477 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:46.039477 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:46.039477 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:46.039477 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:46.039477 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:46.039477 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:46.039477 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:46.039477 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:46.039477 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:46.039477 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 323/640] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 18 16:24:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:46.044906 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d227c0-11f2-4aa2-89c3-e3474a01fba5 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:46.108186 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:46.142401 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b88873-d5e2-42b0-92ba-6dec2dbcb3ff None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:46.152459 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d227c0-11f2-4aa2-89c3-e3474a01fba5 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:46.184843 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90fe1d6-bde2-492f-bad9-d1eeb056b5f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:46.185293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90fe1d6-bde2-492f-bad9-d1eeb056b5f4 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:46.185577 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90fe1d6-bde2-492f-bad9-d1eeb056b5f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:46.185854 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90fe1d6-bde2-492f-bad9-d1eeb056b5f4 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:46.186710 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90fe1d6-bde2-492f-bad9-d1eeb056b5f4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:46.187022 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90fe1d6-bde2-492f-bad9-d1eeb056b5f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:46.188500 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90fe1d6-bde2-492f-bad9-d1eeb056b5f4 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:46.195024 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbed8ba-60e6-45f1-ae2f-f3b3b97f7bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:46.196137 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbed8ba-60e6-45f1-ae2f-f3b3b97f7bee None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:46.196620 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbed8ba-60e6-45f1-ae2f-f3b3b97f7bee None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:46.197102 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbed8ba-60e6-45f1-ae2f-f3b3b97f7bee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:46.207013 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 318/641] 10.222.0.26 () {56 vars in 1137 bytes} [Mon Jul 18 16:24:46 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:46.208521 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a0207c-dd6a-496b-ad76-7d79b3512161 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:46.209000 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a0207c-dd6a-496b-ad76-7d79b3512161 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:46.209431 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a0207c-dd6a-496b-ad76-7d79b3512161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:46.214269 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbed8ba-60e6-45f1-ae2f-f3b3b97f7bee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:46.216184 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbed8ba-60e6-45f1-ae2f-f3b3b97f7bee None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:46.228859 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-50a0207c-dd6a-496b-ad76-7d79b3512161 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:46.230573 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-50a0207c-dd6a-496b-ad76-7d79b3512161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:46.234383 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-0cbed8ba-60e6-45f1-ae2f-f3b3b97f7bee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:46.234383 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:46.234383 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:46.234383 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:46.234383 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:46.234383 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:46.234383 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:46.234383 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:46.234383 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:46.234383 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:46.234383 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:46.234383 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:46.234383 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:46.234383 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:46.234383 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:46.235317 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:46.235317 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:46.235317 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:46.235317 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:46.235317 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:46.235317 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:46.235317 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:46.235317 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:46.235317 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:46.235317 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:46.235317 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:46.235317 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:46.235317 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:46.235317 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:46.235317 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:46.235317 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:46.235317 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:46.241579 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:46.241579 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:46.241579 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:46.241579 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:46.241579 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 324/642] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:46.241579 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbed8ba-60e6-45f1-ae2f-f3b3b97f7bee None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:46.245106 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:46.307544 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 319/643] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:46.307544 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a0207c-dd6a-496b-ad76-7d79b3512161 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:46.361538 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbed8ba-60e6-45f1-ae2f-f3b3b97f7bee None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:46.390017 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:46.407635 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c1df4c-1e34-46d5-8b41-4bb22f529978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:46.407635 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1df4c-1e34-46d5-8b41-4bb22f529978 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:46.407635 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1df4c-1e34-46d5-8b41-4bb22f529978 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:46.407635 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1df4c-1e34-46d5-8b41-4bb22f529978 None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:46.408011 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c1df4c-1e34-46d5-8b41-4bb22f529978 None admin] RBAC: Adding query filter params (name=service) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:46.408011 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c1df4c-1e34-46d5-8b41-4bb22f529978 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:46.408011 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c1df4c-1e34-46d5-8b41-4bb22f529978 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:46.420047 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 325/644] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 18 16:24:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:46.421257 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c1df4c-1e34-46d5-8b41-4bb22f529978 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:46.438808 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a0207c-dd6a-496b-ad76-7d79b3512161 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:46.482535 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3415a3e3-0bde-4b43-9c2e-436972fbb6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:46.483200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415a3e3-0bde-4b43-9c2e-436972fbb6c0 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:46.483484 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415a3e3-0bde-4b43-9c2e-436972fbb6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:46.483786 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415a3e3-0bde-4b43-9c2e-436972fbb6c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:46.506551 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:46.546202 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c1df4c-1e34-46d5-8b41-4bb22f529978 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:46.558465 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3415a3e3-0bde-4b43-9c2e-436972fbb6c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:46.560100 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3415a3e3-0bde-4b43-9c2e-436972fbb6c0 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:46.581342 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1917e957-31bb-41a4-9897-94a1104b7fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:46.582015 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917e957-31bb-41a4-9897-94a1104b7fab None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:46.582305 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917e957-31bb-41a4-9897-94a1104b7fab None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:46.582588 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917e957-31bb-41a4-9897-94a1104b7fab None admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:46.583320 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1917e957-31bb-41a4-9897-94a1104b7fab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:46.583662 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1917e957-31bb-41a4-9897-94a1104b7fab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:46.585046 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1917e957-31bb-41a4-9897-94a1104b7fab None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:46.598202 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 326/645] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:46.599044 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1917e957-31bb-41a4-9897-94a1104b7fab None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:46.626650 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:46.673736 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 320/646] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:24:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:24:46.674606 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3415a3e3-0bde-4b43-9c2e-436972fbb6c0 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:46.717457 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1917e957-31bb-41a4-9897-94a1104b7fab None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:46.751137 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:46.760875 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5551e3ca-3f3f-4055-bff1-4a3d1194e0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:46.761974 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551e3ca-3f3f-4055-bff1-4a3d1194e0dc None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:46.763461 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551e3ca-3f3f-4055-bff1-4a3d1194e0dc None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:46.763461 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551e3ca-3f3f-4055-bff1-4a3d1194e0dc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:46.774191 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5551e3ca-3f3f-4055-bff1-4a3d1194e0dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:46.782935 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-5551e3ca-3f3f-4055-bff1-4a3d1194e0dc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:46.782935 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:46.782935 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:46.782935 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:46.782935 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:46.782935 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:46.782935 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:46.782935 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:46.782935 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:46.782935 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:46.782935 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:46.782935 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:46.782935 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:46.782935 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:46.782935 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:46.782935 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:46.784095 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:46.784095 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:46.784095 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:46.784095 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:46.784095 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:46.784095 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:46.784095 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:46.784095 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:46.784095 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:46.784095 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:46.784095 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:46.784095 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:46.784095 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:46.784095 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:46.784095 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:46.784095 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:46.785324 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:46.785324 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:46.785324 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:46.785324 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:46.785324 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:46.785324 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:46.785324 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:46.785324 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:46.785324 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:46.785324 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:46.785324 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:46.785324 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:46.788202 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 327/647] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 18 16:24:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:46.789074 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5551e3ca-3f3f-4055-bff1-4a3d1194e0dc None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:46.798093 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3415a3e3-0bde-4b43-9c2e-436972fbb6c0 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:46.839105 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398aee11-116f-4372-a934-041d540fc459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:46.840594 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398aee11-116f-4372-a934-041d540fc459 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:46.841105 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398aee11-116f-4372-a934-041d540fc459 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:46.841386 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398aee11-116f-4372-a934-041d540fc459 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:46.852247 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398aee11-116f-4372-a934-041d540fc459 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:46.853772 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398aee11-116f-4372-a934-041d540fc459 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:46.874525 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-398aee11-116f-4372-a934-041d540fc459 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:46.874525 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:46.874525 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:46.874525 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:46.874525 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:46.874525 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:46.874525 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:46.874525 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:46.874525 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:46.874525 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:46.874525 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:46.874525 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:46.874525 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:46.874525 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:46.874525 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:46.875576 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:46.875576 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:46.875576 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:46.875576 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:46.875576 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:46.875576 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:46.875576 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:46.875576 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:46.875576 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:46.875576 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:46.875576 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:46.875576 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:46.875576 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:46.875576 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:46.875576 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:46.875576 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:46.875576 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:46.876491 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:46.876491 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:46.876491 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:46.876491 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:46.880598 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 321/648] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:46.881534 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398aee11-116f-4372-a934-041d540fc459 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:46.894737 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:46.902597 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5551e3ca-3f3f-4055-bff1-4a3d1194e0dc None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:46.943061 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bc1cb1-9f3c-40ff-a56b-d2108f488cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:46.944394 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc1cb1-9f3c-40ff-a56b-d2108f488cf9 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:46.944834 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc1cb1-9f3c-40ff-a56b-d2108f488cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:46.945284 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc1cb1-9f3c-40ff-a56b-d2108f488cf9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:46.953544 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bc1cb1-9f3c-40ff-a56b-d2108f488cf9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:46.955483 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bc1cb1-9f3c-40ff-a56b-d2108f488cf9 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:46.971471 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-a1bc1cb1-9f3c-40ff-a56b-d2108f488cf9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:46.971471 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:46.971471 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:46.971471 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:46.971471 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:46.971471 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:46.971471 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:46.971471 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:46.971471 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:46.971471 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:46.971471 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:46.971471 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:46.971471 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:24:46.971471 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:24:46.971471 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:24:46.972475 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:24:46.972475 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:46.972475 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:46.972475 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:46.972475 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:46.972475 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:46.972475 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:46.972475 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:46.972475 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:46.972475 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:46.972475 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:46.972475 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:46.972475 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:46.972475 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:46.972475 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:46.972475 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:46.972475 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:46.978380 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:24:46.978380 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:24:46.978380 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:24:46.978380 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:24:46.978380 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:24:46.978380 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:24:46.978380 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:46.978380 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:46.978380 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 328/649] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:24:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:46.978380 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bc1cb1-9f3c-40ff-a56b-d2108f488cf9 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:46.994656 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398aee11-116f-4372-a934-041d540fc459 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:47.030050 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d2671-af3b-496c-bd57-0ae072e82780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:47.030271 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:47.030844 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d2671-af3b-496c-bd57-0ae072e82780 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:47.031123 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d2671-af3b-496c-bd57-0ae072e82780 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:47.031395 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d2671-af3b-496c-bd57-0ae072e82780 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:47.041982 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334d2671-af3b-496c-bd57-0ae072e82780 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:47.044149 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334d2671-af3b-496c-bd57-0ae072e82780 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:47.063390 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-334d2671-af3b-496c-bd57-0ae072e82780 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:47.063390 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:47.063390 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:47.063390 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:47.063390 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:47.063390 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:47.063390 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:47.063390 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:47.063390 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:47.063390 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:47.063390 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:47.063390 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:47.063390 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:47.063390 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:47.063390 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:47.064542 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:47.064542 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:47.064542 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:47.064542 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:47.064542 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:47.064542 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:47.064542 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:47.064542 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:47.064542 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:47.064542 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:47.064542 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:47.064542 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:47.064542 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:47.064542 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:47.064542 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:47.064542 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:47.064542 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:47.065439 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:47.065439 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:47.065439 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:47.065439 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:47.065439 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 322/650] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:47.065439 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d2671-af3b-496c-bd57-0ae072e82780 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:47.100326 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bc1cb1-9f3c-40ff-a56b-d2108f488cf9 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:47.137990 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd08e08b-dca1-4f31-aeeb-c1ebe183c238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:47.138675 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd08e08b-dca1-4f31-aeeb-c1ebe183c238 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:47.138960 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd08e08b-dca1-4f31-aeeb-c1ebe183c238 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:47.139239 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd08e08b-dca1-4f31-aeeb-c1ebe183c238 None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:47.140058 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd08e08b-dca1-4f31-aeeb-c1ebe183c238 None admin] RBAC: Adding query filter params (name=service) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:47.140368 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd08e08b-dca1-4f31-aeeb-c1ebe183c238 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:47.141759 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd08e08b-dca1-4f31-aeeb-c1ebe183c238 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:47.144339 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:47.150202 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 329/651] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 18 16:24:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:47.151040 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd08e08b-dca1-4f31-aeeb-c1ebe183c238 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:47.187290 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d2671-af3b-496c-bd57-0ae072e82780 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:47.223323 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07258ab6-f7e9-4f7f-8573-122fd81d7e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:47.223928 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07258ab6-f7e9-4f7f-8573-122fd81d7e9d None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:47.224219 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07258ab6-f7e9-4f7f-8573-122fd81d7e9d None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:47.224497 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07258ab6-f7e9-4f7f-8573-122fd81d7e9d None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:47.225223 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07258ab6-f7e9-4f7f-8573-122fd81d7e9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:47.225530 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07258ab6-f7e9-4f7f-8573-122fd81d7e9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:47.226948 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07258ab6-f7e9-4f7f-8573-122fd81d7e9d None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:47.240241 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 323/652] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:47.241078 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07258ab6-f7e9-4f7f-8573-122fd81d7e9d None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:47.253320 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd08e08b-dca1-4f31-aeeb-c1ebe183c238 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:47.262539 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:47.288048 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d6ce8f-9da6-4482-924d-0411bce1abad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:47.288714 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d6ce8f-9da6-4482-924d-0411bce1abad None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:47.289004 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d6ce8f-9da6-4482-924d-0411bce1abad None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:47.289277 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d6ce8f-9da6-4482-924d-0411bce1abad None admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:47.290064 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d6ce8f-9da6-4482-924d-0411bce1abad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:47.290375 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d6ce8f-9da6-4482-924d-0411bce1abad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:47.291829 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d6ce8f-9da6-4482-924d-0411bce1abad None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:47.305499 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 330/653] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 18 16:24:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:47.306871 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16754285-6942-47be-a87f-f0874921e0ab None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:47.307157 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16754285-6942-47be-a87f-f0874921e0ab None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:47.307438 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16754285-6942-47be-a87f-f0874921e0ab None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:47.308997 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 331/654] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:47.309589 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16754285-6942-47be-a87f-f0874921e0ab None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:47.349256 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07258ab6-f7e9-4f7f-8573-122fd81d7e9d None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:47.374255 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:47.386104 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccebd1f-65d5-4ff5-b05b-c76d5b33e233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:47.390330 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccebd1f-65d5-4ff5-b05b-c76d5b33e233 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:47.390330 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccebd1f-65d5-4ff5-b05b-c76d5b33e233 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:47.390330 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccebd1f-65d5-4ff5-b05b-c76d5b33e233 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:47.390330 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccebd1f-65d5-4ff5-b05b-c76d5b33e233 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:47.390330 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccebd1f-65d5-4ff5-b05b-c76d5b33e233 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:47.390330 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccebd1f-65d5-4ff5-b05b-c76d5b33e233 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:47.401533 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 324/655] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:47.402719 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccebd1f-65d5-4ff5-b05b-c76d5b33e233 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:47.414940 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16754285-6942-47be-a87f-f0874921e0ab None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:47.451395 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae5261-136c-41d0-b4a5-607d26cfbdd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:47.452222 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae5261-136c-41d0-b4a5-607d26cfbdd0 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:47.452505 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae5261-136c-41d0-b4a5-607d26cfbdd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:47.452781 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae5261-136c-41d0-b4a5-607d26cfbdd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:47.462930 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ae5261-136c-41d0-b4a5-607d26cfbdd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:47.464412 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ae5261-136c-41d0-b4a5-607d26cfbdd0 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:47.483483 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-78ae5261-136c-41d0-b4a5-607d26cfbdd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:47.483483 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:47.483483 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:47.483483 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:47.483483 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:47.483483 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:47.483483 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:47.483483 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:47.483483 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:47.483483 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:47.483483 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:47.483483 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:47.483483 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:47.483483 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:47.483483 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:47.484638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:47.484638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:47.484638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:47.484638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:47.484638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:47.484638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:47.484638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:47.484638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:47.484638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:47.484638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:47.484638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:47.484638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:47.484638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:47.484638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:47.484638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:47.484638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:47.484638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:47.485886 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:47.485886 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:47.485886 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:47.485886 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:47.485886 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 332/656] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:47.485886 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae5261-136c-41d0-b4a5-607d26cfbdd0 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:47.508272 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:47.510256 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccebd1f-65d5-4ff5-b05b-c76d5b33e233 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:47.545766 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946133d1-7ff5-45d7-9326-fefcb107a1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:47.546565 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946133d1-7ff5-45d7-9326-fefcb107a1bd None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:47.546848 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946133d1-7ff5-45d7-9326-fefcb107a1bd None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:47.547129 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946133d1-7ff5-45d7-9326-fefcb107a1bd None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:47.555116 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946133d1-7ff5-45d7-9326-fefcb107a1bd None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:47.556608 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946133d1-7ff5-45d7-9326-fefcb107a1bd None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:47.570720 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-946133d1-7ff5-45d7-9326-fefcb107a1bd None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 16:24:47.570720 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:47.570720 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:47.570720 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:47.570720 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:47.570720 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:47.570720 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:47.570720 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:47.570720 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:47.570720 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:47.570720 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:47.570720 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:47.570720 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:24:47.570720 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:24:47.570720 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:24:47.570720 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:24:47.571963 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:47.571963 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:47.571963 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:24:47.571963 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:47.571963 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:24:47.571963 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:47.571963 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:47.571963 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:47.571963 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:47.571963 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:47.571963 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:47.571963 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:47.571963 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:47.571963 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:47.571963 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:47.571963 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:47.571963 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:47.571963 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:47.572924 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:47.572924 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:47.572924 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:24:47.572924 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:24:47.572924 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:24:47.572924 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:24:47.572924 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:24:47.572924 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:24:47.572924 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 16:24:47.572924 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:47.572924 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 325/657] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:24:47 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:47.573567 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4236975-596e-4608-9034-fd1989f88acd None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:47.573567 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4236975-596e-4608-9034-fd1989f88acd None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:47.573688 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4236975-596e-4608-9034-fd1989f88acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:47.592396 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-a4236975-596e-4608-9034-fd1989f88acd None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:47.593598 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a4236975-596e-4608-9034-fd1989f88acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:47.596538 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae5261-136c-41d0-b4a5-607d26cfbdd0 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:47.630033 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:47.631684 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5465842-75f0-4b19-85e6-df3ede2640d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:47.632346 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5465842-75f0-4b19-85e6-df3ede2640d7 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:47.632627 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5465842-75f0-4b19-85e6-df3ede2640d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:47.632901 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5465842-75f0-4b19-85e6-df3ede2640d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:47.633742 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5465842-75f0-4b19-85e6-df3ede2640d7 None admin] RBAC: Adding query filter params (role.id=4fe970e98f394bbfb096c4879bebdf97, scope.project.id=eb9747e2299c4e0991fd7edb4462c7e5, user.id=5b868bfa44b94ca5b470166d39f20e54) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:47.634050 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5465842-75f0-4b19-85e6-df3ede2640d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:47.635482 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5465842-75f0-4b19-85e6-df3ede2640d7 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:47.643040 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 333/658] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 18 16:24:47 2022] GET /identity/v3/role_assignments?user.id=5b868bfa44b94ca5b470166d39f20e54&scope.project.id=eb9747e2299c4e0991fd7edb4462c7e5&role.id=4fe970e98f394bbfb096c4879bebdf97 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:47.644552 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5465842-75f0-4b19-85e6-df3ede2640d7 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:47.671548 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 326/659] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:47.671548 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4236975-596e-4608-9034-fd1989f88acd None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:47.758329 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:47.777740 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5465842-75f0-4b19-85e6-df3ede2640d7 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:47.806254 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4236975-596e-4608-9034-fd1989f88acd None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:47.817332 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42726f7b-4e86-4a48-b5fa-64f44649351a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:47.818510 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42726f7b-4e86-4a48-b5fa-64f44649351a None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:47.819342 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42726f7b-4e86-4a48-b5fa-64f44649351a None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:47.819437 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42726f7b-4e86-4a48-b5fa-64f44649351a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:47.830181 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42726f7b-4e86-4a48-b5fa-64f44649351a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:47.831843 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42726f7b-4e86-4a48-b5fa-64f44649351a None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:47.849747 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-42726f7b-4e86-4a48-b5fa-64f44649351a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:47.849747 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:47.849747 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:47.849747 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:47.849747 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:47.849747 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:47.849747 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:47.849747 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:47.849747 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:47.849747 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:47.849747 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:47.849747 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:47.849747 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:24:47.849747 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:24:47.849747 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:24:47.850688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:24:47.850688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:47.850688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:47.850688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:47.850688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:47.850688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:47.850688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:47.850688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:47.850688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:47.850688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:47.850688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:47.850688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:47.850688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:47.850688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:47.850688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:47.850688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:47.850688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:47.851796 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:24:47.851796 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:24:47.851796 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:24:47.851796 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:24:47.851796 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:24:47.851796 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:24:47.851796 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:47.851796 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:47.857060 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 334/660] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:24:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:47.857060 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42726f7b-4e86-4a48-b5fa-64f44649351a None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:47.857318 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763fef77-57a1-4795-b0e0-42221100cdb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:47.857318 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763fef77-57a1-4795-b0e0-42221100cdb1 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:47.857318 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763fef77-57a1-4795-b0e0-42221100cdb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:47.857318 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763fef77-57a1-4795-b0e0-42221100cdb1 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:47.857698 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763fef77-57a1-4795-b0e0-42221100cdb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:47.858010 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763fef77-57a1-4795-b0e0-42221100cdb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:47.859480 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763fef77-57a1-4795-b0e0-42221100cdb1 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:47.873147 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 327/661] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:47.874105 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763fef77-57a1-4795-b0e0-42221100cdb1 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:47.917227 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:47.972771 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42726f7b-4e86-4a48-b5fa-64f44649351a None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:47.990741 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763fef77-57a1-4795-b0e0-42221100cdb1 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:48.012436 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce379a88-07b6-46cd-84ea-5e1814c4ac0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:48.013443 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce379a88-07b6-46cd-84ea-5e1814c4ac0b None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:48.013871 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce379a88-07b6-46cd-84ea-5e1814c4ac0b None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:48.014308 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce379a88-07b6-46cd-84ea-5e1814c4ac0b None admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:48.015288 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce379a88-07b6-46cd-84ea-5e1814c4ac0b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:48.015898 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce379a88-07b6-46cd-84ea-5e1814c4ac0b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:48.017609 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce379a88-07b6-46cd-84ea-5e1814c4ac0b None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:48.026783 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b7d70-ac32-4da1-a445-27c8db8c153b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:48.027664 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b7d70-ac32-4da1-a445-27c8db8c153b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:48.027965 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b7d70-ac32-4da1-a445-27c8db8c153b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:48.028246 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b7d70-ac32-4da1-a445-27c8db8c153b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:48.041585 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911b7d70-ac32-4da1-a445-27c8db8c153b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:48.041787 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 335/662] 10.222.0.26 () {56 vars in 1137 bytes} [Mon Jul 18 16:24:47 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:48.042789 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce379a88-07b6-46cd-84ea-5e1814c4ac0b None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:48.046175 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:48.046725 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-911b7d70-ac32-4da1-a445-27c8db8c153b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:48.046725 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:48.046725 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:48.046725 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:48.046725 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:48.046725 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:48.046725 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:48.046725 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:48.046725 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:48.046725 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:48.046725 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:48.046725 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:48.046725 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:48.046725 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:48.046725 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:48.046725 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:48.047807 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:48.047807 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:48.047807 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:48.047807 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:48.047807 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:48.047807 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:48.047807 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:48.047807 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:48.047807 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:48.047807 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:48.047807 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:48.047807 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:48.047807 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:48.047807 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:48.047807 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:48.047807 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:48.048689 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:48.048689 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:48.048689 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:48.048689 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:48.048689 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:48.048689 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:48.048689 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:48.048689 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:48.048689 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:48.048689 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:48.048689 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:48.048689 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:48.048689 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 328/663] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 18 16:24:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:48.049454 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b7d70-ac32-4da1-a445-27c8db8c153b None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:48.148314 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:48.156919 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce379a88-07b6-46cd-84ea-5e1814c4ac0b None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:48.157457 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b7d70-ac32-4da1-a445-27c8db8c153b None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:48.195115 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c23ecb-a75f-415c-bdd6-e95e9c989988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:48.195326 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6496b3e-97f4-4858-8e98-18cb9bb983d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:48.195824 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6496b3e-97f4-4858-8e98-18cb9bb983d5 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:48.195996 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c23ecb-a75f-415c-bdd6-e95e9c989988 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:48.196094 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6496b3e-97f4-4858-8e98-18cb9bb983d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:48.196278 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c23ecb-a75f-415c-bdd6-e95e9c989988 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:48.196376 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6496b3e-97f4-4858-8e98-18cb9bb983d5 None admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:48.196560 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c23ecb-a75f-415c-bdd6-e95e9c989988 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:48.197147 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6496b3e-97f4-4858-8e98-18cb9bb983d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:48.197457 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6496b3e-97f4-4858-8e98-18cb9bb983d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:48.198908 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6496b3e-97f4-4858-8e98-18cb9bb983d5 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:48.204977 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c23ecb-a75f-415c-bdd6-e95e9c989988 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:48.206440 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c23ecb-a75f-415c-bdd6-e95e9c989988 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:48.217111 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 336/664] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 18 16:24:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:48.217111 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6496b3e-97f4-4858-8e98-18cb9bb983d5 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:48.226696 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-88c23ecb-a75f-415c-bdd6-e95e9c989988 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 16:24:48.226696 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:48.226696 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:48.226696 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:48.226696 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:48.226696 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:48.226696 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:48.226696 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:48.226696 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:48.226696 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:48.226696 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:48.226696 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:48.226696 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:24:48.226696 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:24:48.226696 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:24:48.226696 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:24:48.227754 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:48.227754 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:48.227754 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:24:48.227754 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:48.227754 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:24:48.227754 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:48.227754 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:48.227754 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:48.227754 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:48.227754 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:48.227754 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:48.227754 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:48.227754 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:48.227754 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:48.227754 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:48.227754 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:48.227754 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:48.227754 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:48.228715 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:48.228715 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:48.228715 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:24:48.228715 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:24:48.228715 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:24:48.228715 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:24:48.228715 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:24:48.228715 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:24:48.228715 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 16:24:48.228715 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:48.228715 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 329/665] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 18 16:24:48 2022] GET /identity/v3/users/neutron => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:48.229328 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c23ecb-a75f-415c-bdd6-e95e9c989988 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:48.288586 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:48.328788 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6496b3e-97f4-4858-8e98-18cb9bb983d5 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:48.339510 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c23ecb-a75f-415c-bdd6-e95e9c989988 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:48.365914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41889c14-0de7-48fc-adee-f5132296ada9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:48.366581 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41889c14-0de7-48fc-adee-f5132296ada9 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:48.366868 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41889c14-0de7-48fc-adee-f5132296ada9 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:48.367150 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41889c14-0de7-48fc-adee-f5132296ada9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:48.376214 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39389ae8-b2d0-472a-8e69-cafd1a09ea60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:48.376930 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39389ae8-b2d0-472a-8e69-cafd1a09ea60 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:48.377219 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39389ae8-b2d0-472a-8e69-cafd1a09ea60 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:48.377493 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39389ae8-b2d0-472a-8e69-cafd1a09ea60 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:48.378243 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39389ae8-b2d0-472a-8e69-cafd1a09ea60 None admin] RBAC: Adding query filter params (name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:48.378550 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39389ae8-b2d0-472a-8e69-cafd1a09ea60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:48.379947 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39389ae8-b2d0-472a-8e69-cafd1a09ea60 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:48.388767 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 330/666] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 18 16:24:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:48.389587 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39389ae8-b2d0-472a-8e69-cafd1a09ea60 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:48.406317 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:48.450195 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41889c14-0de7-48fc-adee-f5132296ada9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:48.452399 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41889c14-0de7-48fc-adee-f5132296ada9 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:48.503665 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39389ae8-b2d0-472a-8e69-cafd1a09ea60 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:48.529095 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:48.543729 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb7e996-3c8d-4a4b-b533-efec33b9aa50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:48.544335 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb7e996-3c8d-4a4b-b533-efec33b9aa50 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:48.544606 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb7e996-3c8d-4a4b-b533-efec33b9aa50 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:48.544887 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb7e996-3c8d-4a4b-b533-efec33b9aa50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:48.545984 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb7e996-3c8d-4a4b-b533-efec33b9aa50 None admin] RBAC: Adding query filter params (role.id=4fe970e98f394bbfb096c4879bebdf97, scope.project.id=eb9747e2299c4e0991fd7edb4462c7e5, user.id=0887908bcb0d4072bf6acb4cbaf1af83) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:48.546714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb7e996-3c8d-4a4b-b533-efec33b9aa50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:48.547831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb7e996-3c8d-4a4b-b533-efec33b9aa50 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:48.555014 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 331/667] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 18 16:24:48 2022] GET /identity/v3/role_assignments?user.id=0887908bcb0d4072bf6acb4cbaf1af83&scope.project.id=eb9747e2299c4e0991fd7edb4462c7e5&role.id=4fe970e98f394bbfb096c4879bebdf97 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:48.555983 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb7e996-3c8d-4a4b-b533-efec33b9aa50 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:48.590452 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 337/668] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:24:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:24:48.592280 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41889c14-0de7-48fc-adee-f5132296ada9 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:48.663429 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:48.688604 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb7e996-3c8d-4a4b-b533-efec33b9aa50 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:48.741166 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41889c14-0de7-48fc-adee-f5132296ada9 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:48.748132 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f44975d-6d4f-45f9-9ad8-2d8b3fedca7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:48.748132 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f44975d-6d4f-45f9-9ad8-2d8b3fedca7e None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:48.748132 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f44975d-6d4f-45f9-9ad8-2d8b3fedca7e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:48.748474 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f44975d-6d4f-45f9-9ad8-2d8b3fedca7e None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:48.749248 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f44975d-6d4f-45f9-9ad8-2d8b3fedca7e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:48.750316 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f44975d-6d4f-45f9-9ad8-2d8b3fedca7e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:48.751137 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f44975d-6d4f-45f9-9ad8-2d8b3fedca7e None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:48.772147 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 332/669] 10.222.0.26 () {56 vars in 1139 bytes} [Mon Jul 18 16:24:48 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:48.773029 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f44975d-6d4f-45f9-9ad8-2d8b3fedca7e None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:48.783040 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab0740-291b-4cc7-896b-e220eb1244a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:48.784000 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab0740-291b-4cc7-896b-e220eb1244a5 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:48.784289 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab0740-291b-4cc7-896b-e220eb1244a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:48.784569 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab0740-291b-4cc7-896b-e220eb1244a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:48.806319 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ab0740-291b-4cc7-896b-e220eb1244a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:48.808223 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ab0740-291b-4cc7-896b-e220eb1244a5 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:48.831013 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-47ab0740-291b-4cc7-896b-e220eb1244a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:48.831013 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:48.831013 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:48.831013 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:48.831013 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:48.831013 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:48.831013 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:48.831013 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:48.831013 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:48.831013 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:48.831013 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:48.831013 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:48.831013 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:48.831013 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:48.831013 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:48.832064 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:48.832064 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:48.832064 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:48.832064 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:48.832064 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:48.832064 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:48.832064 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:48.832064 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:48.832064 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:48.832064 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:48.832064 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:48.832064 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:48.832064 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:48.832064 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:48.832064 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:48.832064 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:48.832064 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:48.832970 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:48.832970 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:48.832970 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:48.832970 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:48.832970 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 338/670] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:48.833678 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab0740-291b-4cc7-896b-e220eb1244a5 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:48.854586 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:48.915313 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f44975d-6d4f-45f9-9ad8-2d8b3fedca7e None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:48.952724 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3242150-9ca9-4d62-bc52-240a6ec5bbd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:48.953906 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3242150-9ca9-4d62-bc52-240a6ec5bbd0 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:48.954386 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3242150-9ca9-4d62-bc52-240a6ec5bbd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:48.954851 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3242150-9ca9-4d62-bc52-240a6ec5bbd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:48.957928 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:48.967281 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3242150-9ca9-4d62-bc52-240a6ec5bbd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:48.967898 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab0740-291b-4cc7-896b-e220eb1244a5 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:48.969235 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3242150-9ca9-4d62-bc52-240a6ec5bbd0 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:48.987249 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-a3242150-9ca9-4d62-bc52-240a6ec5bbd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:48.987249 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:48.987249 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:48.987249 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:48.987249 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:48.987249 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:48.987249 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:48.987249 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:48.987249 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:48.987249 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:48.987249 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:48.987249 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:48.987249 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:48.987249 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:48.987249 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:48.988357 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:48.988357 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:48.988357 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:48.988357 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:48.988357 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:48.988357 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:48.988357 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:48.988357 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:48.988357 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:48.988357 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:48.988357 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:48.988357 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:48.988357 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:48.988357 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:48.988357 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:48.988357 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:48.988357 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:48.989602 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:48.989602 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:48.989602 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:48.989602 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:48.989602 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 333/671] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:48.989602 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3242150-9ca9-4d62-bc52-240a6ec5bbd0 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:49.013887 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7f9325-510e-4e45-b837-3189376135ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:49.014548 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f9325-510e-4e45-b837-3189376135ea None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:49.014827 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f9325-510e-4e45-b837-3189376135ea None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:49.015101 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f9325-510e-4e45-b837-3189376135ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:49.086068 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7f9325-510e-4e45-b837-3189376135ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:49.087482 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7f9325-510e-4e45-b837-3189376135ea None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:49.102527 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:49.108699 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3242150-9ca9-4d62-bc52-240a6ec5bbd0 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:49.148802 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd53c8-9c9d-4eb1-9c1a-5fcda0567aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:49.149717 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd53c8-9c9d-4eb1-9c1a-5fcda0567aa7 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:49.150153 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd53c8-9c9d-4eb1-9c1a-5fcda0567aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:49.150771 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd53c8-9c9d-4eb1-9c1a-5fcda0567aa7 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:49.151778 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbd53c8-9c9d-4eb1-9c1a-5fcda0567aa7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:49.152306 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbd53c8-9c9d-4eb1-9c1a-5fcda0567aa7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:49.153887 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbd53c8-9c9d-4eb1-9c1a-5fcda0567aa7 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:49.167365 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 334/672] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:49.168639 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd53c8-9c9d-4eb1-9c1a-5fcda0567aa7 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:49.197524 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 339/673] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:24:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:24:49.198651 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7f9325-510e-4e45-b837-3189376135ea None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:49.226799 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:49.285725 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd53c8-9c9d-4eb1-9c1a-5fcda0567aa7 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:49.306853 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7f9325-510e-4e45-b837-3189376135ea None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:49.321514 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:49.326940 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97feb33-463b-456e-b87f-7a143d2a659d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:49.327889 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97feb33-463b-456e-b87f-7a143d2a659d None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:49.328343 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97feb33-463b-456e-b87f-7a143d2a659d None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:49.328789 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97feb33-463b-456e-b87f-7a143d2a659d None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:49.349069 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97feb33-463b-456e-b87f-7a143d2a659d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:49.349069 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97feb33-463b-456e-b87f-7a143d2a659d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:49.349069 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97feb33-463b-456e-b87f-7a143d2a659d None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:49.349472 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4261137-7955-486b-bffb-2a250e13698f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:49.349472 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4261137-7955-486b-bffb-2a250e13698f None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:49.349472 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4261137-7955-486b-bffb-2a250e13698f None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:49.349472 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4261137-7955-486b-bffb-2a250e13698f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:49.349762 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 335/674] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:49.349762 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97feb33-463b-456e-b87f-7a143d2a659d None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:49.358074 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4261137-7955-486b-bffb-2a250e13698f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:49.359658 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4261137-7955-486b-bffb-2a250e13698f None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:49.376901 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-c4261137-7955-486b-bffb-2a250e13698f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 16:24:49.376901 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:49.376901 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:49.376901 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:49.376901 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:49.376901 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:49.376901 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:49.376901 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:49.376901 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:49.376901 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:49.376901 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:49.376901 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:49.376901 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:24:49.376901 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:24:49.376901 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:24:49.376901 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:24:49.378126 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:49.378126 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:49.378126 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:24:49.378126 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:49.378126 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:24:49.378126 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:49.378126 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:49.378126 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:49.378126 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:49.378126 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:49.378126 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:49.378126 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:49.378126 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:49.378126 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:49.378126 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:49.378126 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:49.378126 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:49.378126 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:49.379296 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:49.379296 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:49.379296 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:24:49.379296 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:24:49.379296 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:24:49.379296 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:24:49.379296 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:24:49.379296 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:24:49.379296 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 16:24:49.379296 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:49.382032 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 340/675] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 18 16:24:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:49.383018 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4261137-7955-486b-bffb-2a250e13698f None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:49.468792 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97feb33-463b-456e-b87f-7a143d2a659d None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:49.479847 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:49.491318 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4261137-7955-486b-bffb-2a250e13698f None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:49.509883 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97866b1c-0e5b-4448-9f91-33c201fb9341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:49.511030 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97866b1c-0e5b-4448-9f91-33c201fb9341 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:49.511538 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97866b1c-0e5b-4448-9f91-33c201fb9341 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:49.512559 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97866b1c-0e5b-4448-9f91-33c201fb9341 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:49.525481 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97866b1c-0e5b-4448-9f91-33c201fb9341 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:49.527427 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97866b1c-0e5b-4448-9f91-33c201fb9341 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:49.528890 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6c45e4-1b7e-4700-af63-f3e51c5b0391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:49.529751 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c45e4-1b7e-4700-af63-f3e51c5b0391 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:49.530170 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c45e4-1b7e-4700-af63-f3e51c5b0391 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:49.530418 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c45e4-1b7e-4700-af63-f3e51c5b0391 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:49.537739 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6c45e4-1b7e-4700-af63-f3e51c5b0391 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:49.539249 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6c45e4-1b7e-4700-af63-f3e51c5b0391 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:49.548879 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-97866b1c-0e5b-4448-9f91-33c201fb9341 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:49.548879 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:49.548879 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:49.548879 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:49.548879 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:49.548879 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:49.548879 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:49.548879 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:49.548879 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:49.548879 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:49.548879 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:49.548879 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:49.548879 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:49.548879 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:49.548879 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:49.550108 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:49.550108 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:49.550108 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:49.550108 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:49.550108 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:49.550108 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:49.550108 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:49.550108 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:49.550108 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:49.550108 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:49.550108 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:49.550108 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:49.550108 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:49.550108 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:49.550108 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:49.550108 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:49.550108 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:49.551004 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:49.551004 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:49.551004 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:49.551004 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:49.553201 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 336/676] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:49.554193 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97866b1c-0e5b-4448-9f91-33c201fb9341 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:49.560141 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-cb6c45e4-1b7e-4700-af63-f3e51c5b0391 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:49.560141 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:49.560141 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:49.560141 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:49.560141 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:49.560141 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:49.560141 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:49.560141 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:49.560141 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:49.560141 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:49.560141 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:49.560141 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:49.560141 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:24:49.560141 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:24:49.560141 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:24:49.561233 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:24:49.561233 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:49.561233 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:49.561233 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:49.561233 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:49.561233 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:49.561233 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:49.561233 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:49.561233 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:49.561233 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:49.561233 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:49.561233 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:49.561233 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:49.561233 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:49.561233 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:49.561233 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:49.561233 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:49.562368 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:24:49.562368 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:24:49.562368 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:24:49.562368 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:24:49.562368 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:24:49.562368 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:24:49.562368 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:49.562368 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:49.568747 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 341/677] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:24:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:49.569807 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6c45e4-1b7e-4700-af63-f3e51c5b0391 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:49.604841 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:49.674725 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6c45e4-1b7e-4700-af63-f3e51c5b0391 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:49.676165 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97866b1c-0e5b-4448-9f91-33c201fb9341 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:49.712651 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1841d999-5080-4940-8e5f-c183e61b7cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:49.713548 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841d999-5080-4940-8e5f-c183e61b7cad None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:49.713922 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841d999-5080-4940-8e5f-c183e61b7cad None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:49.714379 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841d999-5080-4940-8e5f-c183e61b7cad None admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:49.715356 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1841d999-5080-4940-8e5f-c183e61b7cad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:49.715842 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1841d999-5080-4940-8e5f-c183e61b7cad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:49.717462 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1841d999-5080-4940-8e5f-c183e61b7cad None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:49.718408 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4478a83d-f004-4b88-8a1b-4be1200791a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:49.719779 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478a83d-f004-4b88-8a1b-4be1200791a7 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:49.720210 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478a83d-f004-4b88-8a1b-4be1200791a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:49.720709 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478a83d-f004-4b88-8a1b-4be1200791a7 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:49.721706 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4478a83d-f004-4b88-8a1b-4be1200791a7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:49.722165 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4478a83d-f004-4b88-8a1b-4be1200791a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:49.723768 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4478a83d-f004-4b88-8a1b-4be1200791a7 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:49.735930 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 342/678] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:49.737096 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1841d999-5080-4940-8e5f-c183e61b7cad None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:49.745394 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 337/679] 10.222.0.26 () {56 vars in 1135 bytes} [Mon Jul 18 16:24:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:49.747144 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4478a83d-f004-4b88-8a1b-4be1200791a7 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:49.768471 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:49.853759 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:49.862338 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1841d999-5080-4940-8e5f-c183e61b7cad None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:49.862869 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4478a83d-f004-4b88-8a1b-4be1200791a7 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:49.898758 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c07409-f32c-49a0-9a9d-430ebd609c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:49.899135 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073e811b-fa2f-4a63-9f4a-f23b9bf05488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:49.899610 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c07409-f32c-49a0-9a9d-430ebd609c4c None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:49.899837 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e811b-fa2f-4a63-9f4a-f23b9bf05488 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:49.899959 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c07409-f32c-49a0-9a9d-430ebd609c4c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:49.900125 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e811b-fa2f-4a63-9f4a-f23b9bf05488 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:49.900220 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c07409-f32c-49a0-9a9d-430ebd609c4c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:49.900403 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e811b-fa2f-4a63-9f4a-f23b9bf05488 None admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:49.901196 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073e811b-fa2f-4a63-9f4a-f23b9bf05488 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:49.901508 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073e811b-fa2f-4a63-9f4a-f23b9bf05488 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:49.903695 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073e811b-fa2f-4a63-9f4a-f23b9bf05488 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:49.908184 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c07409-f32c-49a0-9a9d-430ebd609c4c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:49.909641 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c07409-f32c-49a0-9a9d-430ebd609c4c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:49.921229 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 343/680] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 18 16:24:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:49.921229 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073e811b-fa2f-4a63-9f4a-f23b9bf05488 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:49.924831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-b3c07409-f32c-49a0-9a9d-430ebd609c4c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:49.924831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:49.924831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:49.924831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:49.924831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:49.924831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:49.924831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:49.924831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:49.924831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:49.924831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:49.924831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:49.924831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:49.924831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:24:49.924831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:24:49.924831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:24:49.925756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:24:49.925756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:49.925756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:49.925756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:49.925756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:49.925756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:49.925756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:49.925756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:49.925756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:49.925756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:49.925756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:49.925756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:49.925756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:49.925756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:49.925756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:49.925756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:49.925756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:49.926739 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:24:49.926739 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:24:49.926739 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:24:49.926739 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:24:49.926739 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:24:49.926739 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:24:49.926739 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:49.926739 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:49.926739 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 338/681] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:24:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:49.927571 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf67c9b4-22fc-4290-b101-a7bfd502156e None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:49.927761 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf67c9b4-22fc-4290-b101-a7bfd502156e None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:49.928042 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf67c9b4-22fc-4290-b101-a7bfd502156e None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:49.929604 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 339/682] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:49.930316 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf67c9b4-22fc-4290-b101-a7bfd502156e None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:50.015619 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:50.035798 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073e811b-fa2f-4a63-9f4a-f23b9bf05488 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:50.038461 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf67c9b4-22fc-4290-b101-a7bfd502156e None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:50.071452 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59632802-0bbc-4dc4-89e2-9ceb6744832f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:50.072145 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59632802-0bbc-4dc4-89e2-9ceb6744832f None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:50.072429 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59632802-0bbc-4dc4-89e2-9ceb6744832f None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:50.072709 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59632802-0bbc-4dc4-89e2-9ceb6744832f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:50.074053 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bbb66d-fc68-4eb2-9c84-5ef9083845d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:50.075320 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bbb66d-fc68-4eb2-9c84-5ef9083845d3 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:50.075690 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bbb66d-fc68-4eb2-9c84-5ef9083845d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:50.076038 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bbb66d-fc68-4eb2-9c84-5ef9083845d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:50.076873 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bbb66d-fc68-4eb2-9c84-5ef9083845d3 None admin] RBAC: Adding query filter params (role.id=4fe970e98f394bbfb096c4879bebdf97, scope.project.id=eb9747e2299c4e0991fd7edb4462c7e5, user.id=6f2e55df807c43d7951fa4be67ef5e81) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:50.077193 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bbb66d-fc68-4eb2-9c84-5ef9083845d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:50.078635 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bbb66d-fc68-4eb2-9c84-5ef9083845d3 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:50.095523 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 340/683] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 18 16:24:49 2022] GET /identity/v3/role_assignments?user.id=6f2e55df807c43d7951fa4be67ef5e81&scope.project.id=eb9747e2299c4e0991fd7edb4462c7e5&role.id=4fe970e98f394bbfb096c4879bebdf97 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:50.095523 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bbb66d-fc68-4eb2-9c84-5ef9083845d3 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:50.111190 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:50.165594 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59632802-0bbc-4dc4-89e2-9ceb6744832f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:50.167182 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59632802-0bbc-4dc4-89e2-9ceb6744832f None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:50.245590 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bbb66d-fc68-4eb2-9c84-5ef9083845d3 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:50.292825 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f5bdd2-5c5e-412f-a50c-668ff62ab41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:50.293874 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 344/684] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:24:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:24:50.294318 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f5bdd2-5c5e-412f-a50c-668ff62ab41a None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:50.294741 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f5bdd2-5c5e-412f-a50c-668ff62ab41a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:50.295030 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f5bdd2-5c5e-412f-a50c-668ff62ab41a None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:50.296246 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f5bdd2-5c5e-412f-a50c-668ff62ab41a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:50.296565 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f5bdd2-5c5e-412f-a50c-668ff62ab41a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:50.296831 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac545228-cee0-4f1d-9ba8-6eb008113eee None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:50.297366 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac545228-cee0-4f1d-9ba8-6eb008113eee None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:50.297839 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac545228-cee0-4f1d-9ba8-6eb008113eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:50.298104 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f5bdd2-5c5e-412f-a50c-668ff62ab41a None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:50.314454 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 341/685] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 18 16:24:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:50.315407 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f5bdd2-5c5e-412f-a50c-668ff62ab41a None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:50.317271 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-ac545228-cee0-4f1d-9ba8-6eb008113eee None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:50.318685 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-ac545228-cee0-4f1d-9ba8-6eb008113eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:50.328800 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:50.389535 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:50.412689 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 345/686] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:50.419590 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac545228-cee0-4f1d-9ba8-6eb008113eee None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:50.465736 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f5bdd2-5c5e-412f-a50c-668ff62ab41a None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:50.509985 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dd1862-ccc2-4e31-9465-4e860614473d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:50.510788 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd1862-ccc2-4e31-9465-4e860614473d None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:50.511074 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd1862-ccc2-4e31-9465-4e860614473d None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:50.511351 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd1862-ccc2-4e31-9465-4e860614473d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:50.524065 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dd1862-ccc2-4e31-9465-4e860614473d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:50.525708 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dd1862-ccc2-4e31-9465-4e860614473d None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:50.547642 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac545228-cee0-4f1d-9ba8-6eb008113eee None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:50.552366 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-36dd1862-ccc2-4e31-9465-4e860614473d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:50.552366 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:50.552366 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:50.552366 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:50.552366 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:50.552366 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:50.552366 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:50.552366 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:50.552366 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:50.552366 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:50.552366 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:50.552366 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:50.552366 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:50.552366 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:50.552366 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:50.553552 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:50.553552 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:50.553552 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:50.553552 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:50.553552 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:50.553552 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:50.553552 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:50.553552 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:50.553552 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:50.553552 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:50.553552 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:50.553552 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:50.553552 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:50.553552 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:50.553552 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:50.553552 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:50.553552 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:50.554444 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:50.554444 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:50.554444 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:50.554444 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:50.559539 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 342/687] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:50.561158 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dd1862-ccc2-4e31-9465-4e860614473d None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:50.596431 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee67ae-0a71-415a-adaf-4afb55b75ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:50.597668 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee67ae-0a71-415a-adaf-4afb55b75ea0 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:50.598386 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee67ae-0a71-415a-adaf-4afb55b75ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:50.598831 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee67ae-0a71-415a-adaf-4afb55b75ea0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:50.600707 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbee67ae-0a71-415a-adaf-4afb55b75ea0 None admin] RBAC: Adding query filter params (role.id=4fe970e98f394bbfb096c4879bebdf97, scope.project.id=eb9747e2299c4e0991fd7edb4462c7e5, user.id=7a6054dbfbbc487794d2a32061b0ac82) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:50.601091 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbee67ae-0a71-415a-adaf-4afb55b75ea0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:50.603277 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbee67ae-0a71-415a-adaf-4afb55b75ea0 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:50.619532 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 346/688] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 18 16:24:50 2022] GET /identity/v3/role_assignments?user.id=7a6054dbfbbc487794d2a32061b0ac82&scope.project.id=eb9747e2299c4e0991fd7edb4462c7e5&role.id=4fe970e98f394bbfb096c4879bebdf97 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:50.619532 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee67ae-0a71-415a-adaf-4afb55b75ea0 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:50.653450 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:50.682376 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:50.751062 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee67ae-0a71-415a-adaf-4afb55b75ea0 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:50.785616 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dd1862-ccc2-4e31-9465-4e860614473d None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:50.794978 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6233f674-a82c-4b78-941e-27f190b68f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:50.795965 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233f674-a82c-4b78-941e-27f190b68f7f None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:50.796252 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233f674-a82c-4b78-941e-27f190b68f7f None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:50.796527 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233f674-a82c-4b78-941e-27f190b68f7f None admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:50.797277 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6233f674-a82c-4b78-941e-27f190b68f7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:50.797586 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6233f674-a82c-4b78-941e-27f190b68f7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:50.799039 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6233f674-a82c-4b78-941e-27f190b68f7f None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:50.815534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 347/689] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:50.825313 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6233f674-a82c-4b78-941e-27f190b68f7f None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:50.871327 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22366f17-2e93-461d-b965-0aebe799dd7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:50.872593 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22366f17-2e93-461d-b965-0aebe799dd7a None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:50.873104 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22366f17-2e93-461d-b965-0aebe799dd7a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:50.873629 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22366f17-2e93-461d-b965-0aebe799dd7a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:50.889715 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22366f17-2e93-461d-b965-0aebe799dd7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:50.897821 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-22366f17-2e93-461d-b965-0aebe799dd7a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:24:50.897821 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:50.897821 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:50.897821 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:50.897821 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:50.897821 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:50.897821 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:50.897821 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:50.897821 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:50.897821 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:50.897821 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:50.897821 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:50.897821 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:50.897821 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:50.897821 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:50.897821 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:50.898797 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:50.898797 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:50.898797 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:50.898797 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:50.898797 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:50.898797 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:50.898797 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:50.898797 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:50.898797 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:50.898797 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:50.898797 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:50.898797 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:50.898797 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:50.898797 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:50.898797 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:50.898797 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:50.899672 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:50.899672 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:50.899672 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:50.899672 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:50.899672 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:50.899672 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:50.899672 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:50.899672 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:50.899672 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:50.899672 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:50.899672 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:24:50.899672 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:50.907919 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 343/690] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 18 16:24:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:50.907919 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5857d6da-d2e3-438d-ab35-5c2c1af60ae6 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:50.907919 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5857d6da-d2e3-438d-ab35-5c2c1af60ae6 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:50.907919 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5857d6da-d2e3-438d-ab35-5c2c1af60ae6 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:50.907919 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 344/691] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:50.912279 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5857d6da-d2e3-438d-ab35-5c2c1af60ae6 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:50.976604 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6233f674-a82c-4b78-941e-27f190b68f7f None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:50.988428 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:51.016450 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28e6c94-41af-4be7-8862-370465721076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:51.017520 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e6c94-41af-4be7-8862-370465721076 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:51.018330 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e6c94-41af-4be7-8862-370465721076 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:51.019205 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e6c94-41af-4be7-8862-370465721076 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:51.031530 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28e6c94-41af-4be7-8862-370465721076 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:51.031530 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28e6c94-41af-4be7-8862-370465721076 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:51.047180 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-f28e6c94-41af-4be7-8862-370465721076 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:51.047180 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:51.047180 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:51.047180 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:51.047180 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:51.047180 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:51.047180 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:51.047180 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:51.047180 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:51.047180 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:51.047180 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:51.047180 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:51.047180 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:24:51.047180 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:24:51.047180 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:24:51.048387 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:24:51.048387 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:51.048387 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:51.048387 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:51.048387 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:51.048387 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:51.048387 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:51.048387 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:51.048387 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:51.048387 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:51.048387 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:51.048387 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:51.048387 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:51.048387 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:51.048387 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:51.048387 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:51.048387 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:51.049444 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:24:51.049444 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:24:51.049444 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:24:51.049444 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:24:51.049444 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:24:51.049444 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:24:51.049444 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:51.049444 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:51.053311 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 348/692] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:24:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:51.054772 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab285a87-f226-41b7-a557-74f8e0f3c861 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:51.055307 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab285a87-f226-41b7-a557-74f8e0f3c861 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:51.055307 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab285a87-f226-41b7-a557-74f8e0f3c861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:51.065844 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5857d6da-d2e3-438d-ab35-5c2c1af60ae6 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:51.073455 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-ab285a87-f226-41b7-a557-74f8e0f3c861 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:51.075201 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-ab285a87-f226-41b7-a557-74f8e0f3c861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:51.081909 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:51.103975 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4f1179-33cd-439d-819f-affa85bd0650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:51.104893 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f1179-33cd-439d-819f-affa85bd0650 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:51.105350 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f1179-33cd-439d-819f-affa85bd0650 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:51.106311 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f1179-33cd-439d-819f-affa85bd0650 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:51.107206 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4f1179-33cd-439d-819f-affa85bd0650 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:51.107745 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4f1179-33cd-439d-819f-affa85bd0650 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:51.109305 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4f1179-33cd-439d-819f-affa85bd0650 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:51.124496 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 345/693] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:24:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:51.125651 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4f1179-33cd-439d-819f-affa85bd0650 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:51.156376 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 349/694] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:51.157297 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab285a87-f226-41b7-a557-74f8e0f3c861 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:51.241575 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4f1179-33cd-439d-819f-affa85bd0650 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:51.264679 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:51.269602 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab285a87-f226-41b7-a557-74f8e0f3c861 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:51.280623 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adac5b9-9407-486e-ba91-f054b2dd3458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:51.281637 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adac5b9-9407-486e-ba91-f054b2dd3458 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:51.282085 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adac5b9-9407-486e-ba91-f054b2dd3458 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:51.282508 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adac5b9-9407-486e-ba91-f054b2dd3458 None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:51.283450 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adac5b9-9407-486e-ba91-f054b2dd3458 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:51.283931 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adac5b9-9407-486e-ba91-f054b2dd3458 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:51.285539 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adac5b9-9407-486e-ba91-f054b2dd3458 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:51.299447 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 346/695] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 18 16:24:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:51.300628 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adac5b9-9407-486e-ba91-f054b2dd3458 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:51.321245 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4c15f4-bd77-4996-bc7e-592fc7de1b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:51.322870 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c15f4-bd77-4996-bc7e-592fc7de1b9f None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:51.323473 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c15f4-bd77-4996-bc7e-592fc7de1b9f None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:51.323928 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c15f4-bd77-4996-bc7e-592fc7de1b9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:51.338514 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4c15f4-bd77-4996-bc7e-592fc7de1b9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:51.340911 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4c15f4-bd77-4996-bc7e-592fc7de1b9f None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:51.357630 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:51.372567 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-3d4c15f4-bd77-4996-bc7e-592fc7de1b9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:51.372567 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:51.372567 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:51.372567 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:51.372567 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:51.372567 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:51.372567 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:51.372567 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:51.372567 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:51.372567 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:51.372567 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:51.372567 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:51.372567 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:51.372567 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:51.372567 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:51.374064 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:51.374064 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:51.374064 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:51.374064 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:51.374064 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:51.374064 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:51.374064 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:51.374064 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:51.374064 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:51.374064 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:51.374064 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:51.374064 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:51.374064 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:51.374064 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:51.374064 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:51.374064 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:51.374064 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:51.375292 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:51.375292 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:51.375292 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:51.375292 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:51.378780 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 350/696] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:51.380226 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4c15f4-bd77-4996-bc7e-592fc7de1b9f None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:51.459618 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adac5b9-9407-486e-ba91-f054b2dd3458 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:51.501669 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39d1364-5476-4900-a73f-6484cf370320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:51.502530 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39d1364-5476-4900-a73f-6484cf370320 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:51.502815 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39d1364-5476-4900-a73f-6484cf370320 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:51.503149 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39d1364-5476-4900-a73f-6484cf370320 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:51.516453 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39d1364-5476-4900-a73f-6484cf370320 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:51.517918 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39d1364-5476-4900-a73f-6484cf370320 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:51.537371 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4c15f4-bd77-4996-bc7e-592fc7de1b9f None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:51.540483 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-e39d1364-5476-4900-a73f-6484cf370320 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:51.540483 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:51.540483 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:51.540483 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:51.540483 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:51.540483 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:51.540483 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:51.540483 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:51.540483 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:51.540483 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:51.540483 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:51.540483 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:51.540483 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:51.540483 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:51.540483 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:51.542070 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:51.542070 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:51.542070 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:51.542070 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:51.542070 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:51.542070 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:51.542070 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:51.542070 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:51.542070 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:51.542070 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:51.542070 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:51.542070 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:51.542070 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:51.542070 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:51.542070 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:51.542070 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:51.542070 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:51.542998 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:51.542998 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:51.542998 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:51.542998 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:51.542998 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 347/697] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:51.542998 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39d1364-5476-4900-a73f-6484cf370320 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:51.577651 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f996ffa1-5a20-425b-9e1a-3a315e38b804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:51.578343 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f996ffa1-5a20-425b-9e1a-3a315e38b804 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:51.578705 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f996ffa1-5a20-425b-9e1a-3a315e38b804 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:51.578984 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f996ffa1-5a20-425b-9e1a-3a315e38b804 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:51.579871 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f996ffa1-5a20-425b-9e1a-3a315e38b804 None admin] RBAC: Adding query filter params (role.id=4fe970e98f394bbfb096c4879bebdf97, scope.project.id=eb9747e2299c4e0991fd7edb4462c7e5, user.id=43854d7a6df2466eb7976fabe6011767) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:51.580187 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f996ffa1-5a20-425b-9e1a-3a315e38b804 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:51.581876 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f996ffa1-5a20-425b-9e1a-3a315e38b804 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:51.588977 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 351/698] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 18 16:24:51 2022] GET /identity/v3/role_assignments?user.id=43854d7a6df2466eb7976fabe6011767&scope.project.id=eb9747e2299c4e0991fd7edb4462c7e5&role.id=4fe970e98f394bbfb096c4879bebdf97 => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:51.590339 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f996ffa1-5a20-425b-9e1a-3a315e38b804 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:51.610794 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:51.654472 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:51.669116 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39d1364-5476-4900-a73f-6484cf370320 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:51.714322 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c9ca61-596f-46ca-801e-2f65d2f9e802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:51.715127 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c9ca61-596f-46ca-801e-2f65d2f9e802 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:51.715414 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c9ca61-596f-46ca-801e-2f65d2f9e802 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:51.716035 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c9ca61-596f-46ca-801e-2f65d2f9e802 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:51.716842 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c9ca61-596f-46ca-801e-2f65d2f9e802 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:51.717150 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c9ca61-596f-46ca-801e-2f65d2f9e802 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:51.718597 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c9ca61-596f-46ca-801e-2f65d2f9e802 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:51.718771 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f996ffa1-5a20-425b-9e1a-3a315e38b804 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:51.737242 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 348/699] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:51.738318 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c9ca61-596f-46ca-801e-2f65d2f9e802 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:51.758561 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51323df-ccd5-474e-abcd-abce7f07e6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:51.759283 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51323df-ccd5-474e-abcd-abce7f07e6cb None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:51.759607 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51323df-ccd5-474e-abcd-abce7f07e6cb None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:51.759922 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51323df-ccd5-474e-abcd-abce7f07e6cb None admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:51.760741 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51323df-ccd5-474e-abcd-abce7f07e6cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:51.761050 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51323df-ccd5-474e-abcd-abce7f07e6cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:51.763073 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51323df-ccd5-474e-abcd-abce7f07e6cb None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:51.776688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 352/700] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:51.784588 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51323df-ccd5-474e-abcd-abce7f07e6cb None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:51.880621 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:51.881544 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c9ca61-596f-46ca-801e-2f65d2f9e802 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:51.908020 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51323df-ccd5-474e-abcd-abce7f07e6cb None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:51.934470 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8baea2-3b1e-4c2e-ad3f-70a224911619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:51.935443 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8baea2-3b1e-4c2e-ad3f-70a224911619 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:51.935781 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8baea2-3b1e-4c2e-ad3f-70a224911619 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:51.936059 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8baea2-3b1e-4c2e-ad3f-70a224911619 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:51.945586 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8baea2-3b1e-4c2e-ad3f-70a224911619 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:51.947115 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8baea2-3b1e-4c2e-ad3f-70a224911619 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:51.948295 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e6e3c6-cdf1-4d31-bf12-a67286a8eb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:51.949102 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e6e3c6-cdf1-4d31-bf12-a67286a8eb25 None admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:51.949504 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e6e3c6-cdf1-4d31-bf12-a67286a8eb25 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:51.949904 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e6e3c6-cdf1-4d31-bf12-a67286a8eb25 None admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:51.950626 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e6e3c6-cdf1-4d31-bf12-a67286a8eb25 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:51.952134 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e6e3c6-cdf1-4d31-bf12-a67286a8eb25 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:51.963080 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-3c8baea2-3b1e-4c2e-ad3f-70a224911619 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:24:51.963080 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:51.963080 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:51.963080 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:51.963080 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:51.963080 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:51.963080 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:51.963080 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:51.963080 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:51.963080 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:51.963080 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:51.963080 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:51.963080 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:24:51.963080 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:24:51.963080 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:24:51.963080 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:24:51.964116 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:51.964116 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:51.964116 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:24:51.964116 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:51.964116 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:24:51.964116 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:51.964116 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:51.964116 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:51.964116 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:51.964116 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:51.964116 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:51.964116 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:51.964116 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:51.964116 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:51.964116 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:51.964116 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:51.964116 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:51.964116 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:51.965123 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:51.965123 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:51.965123 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:24:51.965123 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:24:51.965123 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:24:51.965123 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:24:51.965123 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:24:51.965123 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:24:51.965123 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:24:51.965123 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:51.965123 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 349/701] 10.222.0.26 () {56 vars in 1093 bytes} [Mon Jul 18 16:24:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:51.970496 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8baea2-3b1e-4c2e-ad3f-70a224911619 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:51.983458 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:52.084039 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 353/702] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 18 16:24:51 2022] POST /identity/v3/users => generated 254 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:24:52.092771 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8baea2-3b1e-4c2e-ad3f-70a224911619 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:52.134094 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a9e5b8-c2e5-43e7-9d43-7896493f3cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:52.134832 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9e5b8-c2e5-43e7-9d43-7896493f3cf5 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:52.135165 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9e5b8-c2e5-43e7-9d43-7896493f3cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:52.135447 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9e5b8-c2e5-43e7-9d43-7896493f3cf5 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:52.136351 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a9e5b8-c2e5-43e7-9d43-7896493f3cf5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:52.137926 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a9e5b8-c2e5-43e7-9d43-7896493f3cf5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:52.144591 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a9e5b8-c2e5-43e7-9d43-7896493f3cf5 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:52.167648 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 350/703] 10.222.0.26 () {56 vars in 1133 bytes} [Mon Jul 18 16:24:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:52.173975 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e6e3c6-cdf1-4d31-bf12-a67286a8eb25 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:52.346094 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e6e3c6-cdf1-4d31-bf12-a67286a8eb25 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:52.378660 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:52.385242 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cac084-7f9b-43a6-8ef7-b69188326c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:52.385910 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cac084-7f9b-43a6-8ef7-b69188326c56 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:52.386192 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cac084-7f9b-43a6-8ef7-b69188326c56 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:52.386469 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cac084-7f9b-43a6-8ef7-b69188326c56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:52.464124 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cac084-7f9b-43a6-8ef7-b69188326c56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:52.466032 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cac084-7f9b-43a6-8ef7-b69188326c56 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:52.585103 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 354/704] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:24:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:24:52.591480 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a9e5b8-c2e5-43e7-9d43-7896493f3cf5 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:52.604263 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355584cf-dfe1-4254-826f-245fe25c11a6 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:52.604263 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355584cf-dfe1-4254-826f-245fe25c11a6 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:52.604521 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355584cf-dfe1-4254-826f-245fe25c11a6 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:52.606607 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 355/705] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:24:52.611858 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d0a7f9-3453-4a57-a699-08d310692dbf None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:52.612148 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d0a7f9-3453-4a57-a699-08d310692dbf None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:52.612430 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d0a7f9-3453-4a57-a699-08d310692dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:52.630123 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-80d0a7f9-3453-4a57-a699-08d310692dbf None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:52.631315 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-80d0a7f9-3453-4a57-a699-08d310692dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:52.664065 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:52.723865 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:52.749814 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 356/706] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:24:52.781525 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a9e5b8-c2e5-43e7-9d43-7896493f3cf5 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:52.783585 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d0a7f9-3453-4a57-a699-08d310692dbf None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:52.879654 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf266d37-8595-4ef9-8694-880f69123c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:52.880368 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf266d37-8595-4ef9-8694-880f69123c3c None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:52.880645 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf266d37-8595-4ef9-8694-880f69123c3c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:52.880912 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf266d37-8595-4ef9-8694-880f69123c3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:52.903347 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf266d37-8595-4ef9-8694-880f69123c3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:52.904838 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf266d37-8595-4ef9-8694-880f69123c3c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:52.933259 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d0a7f9-3453-4a57-a699-08d310692dbf None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:52.943677 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-bf266d37-8595-4ef9-8694-880f69123c3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:52.943677 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:52.943677 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:52.943677 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:52.943677 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:52.943677 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:52.943677 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:52.943677 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:52.943677 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:52.943677 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:52.943677 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:52.943677 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:52.943677 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:52.943677 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:52.943677 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:52.944617 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:52.944617 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:52.944617 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:52.944617 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:52.944617 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:52.944617 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:52.944617 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:52.944617 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:52.944617 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:52.944617 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:52.944617 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:52.944617 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:52.944617 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:52.944617 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:52.944617 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:52.944617 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:52.944617 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:52.945667 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:52.945667 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:52.945667 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:52.945667 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:52.945667 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 351/707] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:52.952616 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf266d37-8595-4ef9-8694-880f69123c3c None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:52.994898 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425105d1-ba4b-4b2d-8906-900b0e4a4355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:52.995741 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425105d1-ba4b-4b2d-8906-900b0e4a4355 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:52.996022 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425105d1-ba4b-4b2d-8906-900b0e4a4355 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:52.996857 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425105d1-ba4b-4b2d-8906-900b0e4a4355 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:52.997517 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425105d1-ba4b-4b2d-8906-900b0e4a4355 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:52.998891 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425105d1-ba4b-4b2d-8906-900b0e4a4355 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:53.013513 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-425105d1-ba4b-4b2d-8906-900b0e4a4355 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 16:24:53.013513 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:53.013513 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:24:53.013513 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:24:53.013513 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:24:53.013513 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:24:53.013513 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:24:53.013513 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:24:53.013513 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 16:24:53.013513 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:53.013513 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:24:53.013513 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:53.013513 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:53.013513 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:53.013513 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:53.013513 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:53.014524 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:53.014524 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:53.014524 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:53.014524 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:53.014524 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:53.014524 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:53.014524 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:53.014524 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:24:53.014524 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:24:53.014524 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:24:53.014524 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:24:53.014524 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:53.014524 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:53.014524 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:53.014524 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:53.014524 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:53.015836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:53.015836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:53.015836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:53.015836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:53.015836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:53.015836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:53.015836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:53.015836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:53.015836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:53.015836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:53.015836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:53.015836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:24:53.015836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:24:53.015836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 16:24:53.015836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:53.015836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 357/708] 10.222.0.26 () {56 vars in 1129 bytes} [Mon Jul 18 16:24:52 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:24:53.021247 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425105d1-ba4b-4b2d-8906-900b0e4a4355 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:53.071822 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:53.177403 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425105d1-ba4b-4b2d-8906-900b0e4a4355 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:53.186141 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf266d37-8595-4ef9-8694-880f69123c3c None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:53.220297 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e73679-ea92-41a2-aed4-7bd75bae2dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:53.220992 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e73679-ea92-41a2-aed4-7bd75bae2dac None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:53.221275 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e73679-ea92-41a2-aed4-7bd75bae2dac None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:53.221552 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e73679-ea92-41a2-aed4-7bd75bae2dac None admin] PATH_INFO: `/v3/services` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:53.222275 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e73679-ea92-41a2-aed4-7bd75bae2dac None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:53.222580 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e73679-ea92-41a2-aed4-7bd75bae2dac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:53.224028 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e73679-ea92-41a2-aed4-7bd75bae2dac None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:53.235894 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:53.237228 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 358/709] 10.222.0.26 () {56 vars in 1124 bytes} [Mon Jul 18 16:24:53 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:53.248529 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e73679-ea92-41a2-aed4-7bd75bae2dac None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:53.254202 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8964c19c-4efd-4aae-bf33-497d6ca42478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:53.255228 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8964c19c-4efd-4aae-bf33-497d6ca42478 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:53.255773 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8964c19c-4efd-4aae-bf33-497d6ca42478 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:53.256258 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8964c19c-4efd-4aae-bf33-497d6ca42478 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:53.257229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8964c19c-4efd-4aae-bf33-497d6ca42478 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:53.257740 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8964c19c-4efd-4aae-bf33-497d6ca42478 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:53.259375 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8964c19c-4efd-4aae-bf33-497d6ca42478 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:53.277021 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 352/710] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:53.278684 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8964c19c-4efd-4aae-bf33-497d6ca42478 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:53.382521 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:53.396660 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e73679-ea92-41a2-aed4-7bd75bae2dac None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:53.404923 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8964c19c-4efd-4aae-bf33-497d6ca42478 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:53.441884 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f4186f-2502-411b-a2dc-f685f1e7a21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:53.442133 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2838dc86-7fc3-4b87-a4cc-d6aa77767398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:53.442797 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838dc86-7fc3-4b87-a4cc-d6aa77767398 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:53.443083 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838dc86-7fc3-4b87-a4cc-d6aa77767398 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:53.443359 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838dc86-7fc3-4b87-a4cc-d6aa77767398 None admin] PATH_INFO: `/v3/services` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:53.443712 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f4186f-2502-411b-a2dc-f685f1e7a21e None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:53.444097 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2838dc86-7fc3-4b87-a4cc-d6aa77767398 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:53.444373 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f4186f-2502-411b-a2dc-f685f1e7a21e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:53.444476 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2838dc86-7fc3-4b87-a4cc-d6aa77767398 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:53.444924 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f4186f-2502-411b-a2dc-f685f1e7a21e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:53.445757 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2838dc86-7fc3-4b87-a4cc-d6aa77767398 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:53.455574 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f4186f-2502-411b-a2dc-f685f1e7a21e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:53.455776 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 359/711] 10.222.0.26 () {56 vars in 1124 bytes} [Mon Jul 18 16:24:53 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:53.455776 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37acb39-ce1e-4a3e-8af6-99b9dab6ee1b None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:53.455776 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37acb39-ce1e-4a3e-8af6-99b9dab6ee1b None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:53.455776 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37acb39-ce1e-4a3e-8af6-99b9dab6ee1b None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:53.461395 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f4186f-2502-411b-a2dc-f685f1e7a21e None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:53.461699 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 360/712] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:53.463245 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abc450e-d7e2-49ee-b5fa-3d8658f68931 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:53.463705 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abc450e-d7e2-49ee-b5fa-3d8658f68931 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:53.464130 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abc450e-d7e2-49ee-b5fa-3d8658f68931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:53.498714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-f8f4186f-2502-411b-a2dc-f685f1e7a21e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:53.498714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:53.498714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:53.498714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:53.498714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:53.498714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:53.498714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:53.498714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:53.498714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:53.498714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:53.498714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:53.498714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:53.498714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:24:53.498714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:24:53.498714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:24:53.499893 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:24:53.499893 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:53.499893 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:53.499893 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:53.499893 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:53.499893 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:53.499893 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:53.499893 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:53.499893 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:53.499893 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:53.499893 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:53.499893 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:53.499893 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:53.499893 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:53.499893 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:53.499893 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:53.499893 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:53.501024 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:24:53.501024 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:24:53.501024 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:24:53.501024 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:24:53.501024 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:24:53.501024 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:24:53.501024 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:53.501024 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:53.504920 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-4abc450e-d7e2-49ee-b5fa-3d8658f68931 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:53.505372 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 353/713] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:24:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:24:53.506781 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-4abc450e-d7e2-49ee-b5fa-3d8658f68931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:53.509762 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f4186f-2502-411b-a2dc-f685f1e7a21e None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:53.570565 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:53.646893 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f4186f-2502-411b-a2dc-f685f1e7a21e None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:53.648840 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 361/714] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:53.691245 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39962cb0-629a-487d-b1fa-b9a5069c477e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:53.692275 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39962cb0-629a-487d-b1fa-b9a5069c477e None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:53.692746 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39962cb0-629a-487d-b1fa-b9a5069c477e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:53.693212 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39962cb0-629a-487d-b1fa-b9a5069c477e None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:53.693720 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abc450e-d7e2-49ee-b5fa-3d8658f68931 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:53.694328 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39962cb0-629a-487d-b1fa-b9a5069c477e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:53.694833 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39962cb0-629a-487d-b1fa-b9a5069c477e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:53.697710 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39962cb0-629a-487d-b1fa-b9a5069c477e None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:53.716104 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 354/715] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 18 16:24:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:53.722447 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39962cb0-629a-487d-b1fa-b9a5069c477e None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:53.847864 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:53.877014 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abc450e-d7e2-49ee-b5fa-3d8658f68931 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:53.915910 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:53.919369 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39962cb0-629a-487d-b1fa-b9a5069c477e None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:53.945870 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ee7d89-b5a8-429f-b850-859c9273588b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:53.947532 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ee7d89-b5a8-429f-b850-859c9273588b None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:53.947818 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ee7d89-b5a8-429f-b850-859c9273588b None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:53.948089 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ee7d89-b5a8-429f-b850-859c9273588b None admin] PATH_INFO: `/v3/services/network` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:53.948749 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ee7d89-b5a8-429f-b850-859c9273588b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:53.951247 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ee7d89-b5a8-429f-b850-859c9273588b None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:53.959947 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829b494a-1216-4805-84e1-fd9d1585290a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:53.960691 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b494a-1216-4805-84e1-fd9d1585290a None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:53.960975 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b494a-1216-4805-84e1-fd9d1585290a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:53.961251 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b494a-1216-4805-84e1-fd9d1585290a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:53.962147 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829b494a-1216-4805-84e1-fd9d1585290a None admin] RBAC: Adding query filter params (role.id=98eaadb3669f4b608d572c001742f559, scope.project.id=eb9747e2299c4e0991fd7edb4462c7e5, user.id=5b868bfa44b94ca5b470166d39f20e54) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:53.962454 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829b494a-1216-4805-84e1-fd9d1585290a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:53.963991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829b494a-1216-4805-84e1-fd9d1585290a None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:53.970604 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 355/716] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 18 16:24:53 2022] GET /identity/v3/role_assignments?user.id=5b868bfa44b94ca5b470166d39f20e54&scope.project.id=eb9747e2299c4e0991fd7edb4462c7e5&role.id=98eaadb3669f4b608d572c001742f559 => generated 259 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:53.976454 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-c2ee7d89-b5a8-429f-b850-859c9273588b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 16:24:53.976454 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:53.976454 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:24:53.976454 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:24:53.976454 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:24:53.976454 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:24:53.976454 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:24:53.976454 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:24:53.976454 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 16:24:53.976454 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:53.976454 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:24:53.976454 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:53.976454 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:53.976454 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:53.976454 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:53.976454 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:53.976454 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:53.977537 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:53.977537 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:53.977537 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:53.977537 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:53.977537 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:53.977537 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:53.977537 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:24:53.977537 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:24:53.977537 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:24:53.977537 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:24:53.977537 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:53.977537 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:53.977537 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:53.977537 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:53.977537 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:53.977537 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:53.978505 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:53.978505 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:53.978505 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:53.978505 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:53.978505 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:53.978505 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:53.978505 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:53.978505 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:53.978505 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:53.978505 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:53.978505 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:24:53.978505 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:24:53.978505 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 16:24:53.978505 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:53.978505 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 362/717] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:24:53 2022] GET /identity/v3/services/network => generated 88 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:24:53.984978 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829b494a-1216-4805-84e1-fd9d1585290a None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:54.149683 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829b494a-1216-4805-84e1-fd9d1585290a None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:54.193369 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b54f643-47b6-4ac5-b2db-7604db55ec9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:54.194037 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b54f643-47b6-4ac5-b2db-7604db55ec9e None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:54.194316 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b54f643-47b6-4ac5-b2db-7604db55ec9e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:54.194589 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b54f643-47b6-4ac5-b2db-7604db55ec9e None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:54.195295 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b54f643-47b6-4ac5-b2db-7604db55ec9e None admin] RBAC: Adding query filter params (name=network) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:54.195623 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b54f643-47b6-4ac5-b2db-7604db55ec9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:54.197031 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b54f643-47b6-4ac5-b2db-7604db55ec9e None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:54.207769 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 356/718] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Jul 18 16:24:53 2022] GET /identity/v3/services?name=network => generated 124 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:54.213316 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ee7d89-b5a8-429f-b850-859c9273588b None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:54.352561 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ee7d89-b5a8-429f-b850-859c9273588b None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:54.364225 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1aec2-ce77-4dc0-8410-39b83aece649 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:54.364507 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1aec2-ce77-4dc0-8410-39b83aece649 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:54.364890 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1aec2-ce77-4dc0-8410-39b83aece649 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:54.366268 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 357/719] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:24:54.372259 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23387ce-07d3-4613-a9cf-7584f04df110 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:54.372837 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23387ce-07d3-4613-a9cf-7584f04df110 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:54.373344 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23387ce-07d3-4613-a9cf-7584f04df110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:54.397551 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-e23387ce-07d3-4613-a9cf-7584f04df110 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:54.400038 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e23387ce-07d3-4613-a9cf-7584f04df110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:54.437631 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460e9501-e3f6-4b79-9386-3ac81c9ddb84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:54.439814 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e9501-e3f6-4b79-9386-3ac81c9ddb84 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:54.440651 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e9501-e3f6-4b79-9386-3ac81c9ddb84 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:54.442115 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e9501-e3f6-4b79-9386-3ac81c9ddb84 None admin] PATH_INFO: `/v3/services` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:54.443042 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460e9501-e3f6-4b79-9386-3ac81c9ddb84 None admin] RBAC: Adding query filter params (type=network) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:54.446668 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460e9501-e3f6-4b79-9386-3ac81c9ddb84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:54.448479 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:54.450250 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460e9501-e3f6-4b79-9386-3ac81c9ddb84 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:54.457221 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 363/720] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Jul 18 16:24:54 2022] GET /identity/v3/services?type=network => generated 124 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:54.546835 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 358/721] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:54.591293 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460e9501-e3f6-4b79-9386-3ac81c9ddb84 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:54.612117 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:54.720662 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460e9501-e3f6-4b79-9386-3ac81c9ddb84 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:54.773094 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c893d08-b4cf-4ff1-8118-38ae63c4ad48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:54.774250 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c893d08-b4cf-4ff1-8118-38ae63c4ad48 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:54.774720 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c893d08-b4cf-4ff1-8118-38ae63c4ad48 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:54.775194 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c893d08-b4cf-4ff1-8118-38ae63c4ad48 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:54.789141 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9889f295-ca94-4365-9853-21c4e2494426 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:54.789431 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9889f295-ca94-4365-9853-21c4e2494426 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:54.789715 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9889f295-ca94-4365-9853-21c4e2494426 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:54.791398 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 359/722] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:54.792661 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c893d08-b4cf-4ff1-8118-38ae63c4ad48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:54.800919 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7c484-2d40-47c0-a231-e9a3f75d9afd None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:54.801207 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7c484-2d40-47c0-a231-e9a3f75d9afd None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:54.801486 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7c484-2d40-47c0-a231-e9a3f75d9afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:54.802673 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-7c893d08-b4cf-4ff1-8118-38ae63c4ad48 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:24:54.802673 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:54.802673 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:54.802673 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:54.802673 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:54.802673 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:54.802673 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:54.802673 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:54.802673 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:54.802673 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:54.802673 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:54.802673 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:54.802673 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:54.802673 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:54.802673 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:54.802673 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:54.803617 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:54.803617 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:54.803617 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:54.803617 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:54.803617 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:54.803617 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:54.803617 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:54.803617 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:54.803617 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:54.803617 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:54.803617 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:54.803617 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:54.803617 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:54.803617 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:54.803617 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:54.803617 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:54.804446 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:54.804446 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:54.804446 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:54.804446 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:54.804446 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:54.804446 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:54.804446 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:54.804446 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:54.804446 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:54.804446 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:54.804446 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:24:54.804446 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:54.807162 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 364/723] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 18 16:24:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:54.812529 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c893d08-b4cf-4ff1-8118-38ae63c4ad48 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:54.838599 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-08b7c484-2d40-47c0-a231-e9a3f75d9afd None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:54.848014 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-08b7c484-2d40-47c0-a231-e9a3f75d9afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:54.902149 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:54.938268 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c893d08-b4cf-4ff1-8118-38ae63c4ad48 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:54.962231 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 360/724] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:54.977803 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ede0b81-87ff-4d6b-8e84-0d6c66b8a444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:54.978832 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede0b81-87ff-4d6b-8e84-0d6c66b8a444 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:54.979343 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede0b81-87ff-4d6b-8e84-0d6c66b8a444 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:54.979906 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede0b81-87ff-4d6b-8e84-0d6c66b8a444 None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:54.980989 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ede0b81-87ff-4d6b-8e84-0d6c66b8a444 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:54.981539 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ede0b81-87ff-4d6b-8e84-0d6c66b8a444 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:54.983219 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ede0b81-87ff-4d6b-8e84-0d6c66b8a444 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:54.992992 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 365/725] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:24:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:55.000937 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b7c484-2d40-47c0-a231-e9a3f75d9afd None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:55.004643 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ede0b81-87ff-4d6b-8e84-0d6c66b8a444 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:55.114182 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ede0b81-87ff-4d6b-8e84-0d6c66b8a444 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:55.146160 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:55.151366 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f76ea-925b-46d3-a513-e07c0bbe7763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:55.152511 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f76ea-925b-46d3-a513-e07c0bbe7763 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:55.153004 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f76ea-925b-46d3-a513-e07c0bbe7763 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:55.153482 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f76ea-925b-46d3-a513-e07c0bbe7763 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:55.166097 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4f76ea-925b-46d3-a513-e07c0bbe7763 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:55.169446 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:55.176272 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-5a4f76ea-925b-46d3-a513-e07c0bbe7763 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:55.176272 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:55.176272 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:55.176272 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:55.176272 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:55.176272 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:55.176272 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:55.176272 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:55.176272 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:55.176272 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:55.176272 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:55.176272 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:55.176272 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:55.176272 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:55.176272 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:55.176272 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:55.177400 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:55.177400 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:55.177400 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:55.177400 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:55.177400 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:55.177400 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:55.177400 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:55.177400 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:55.177400 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:55.177400 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:55.177400 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:55.177400 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:55.177400 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:55.177400 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:55.177400 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:55.177400 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:55.178246 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:55.178246 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:55.178246 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:55.178246 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:55.178246 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:55.178246 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:55.178246 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:55.178246 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:55.178246 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:55.178246 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:55.178246 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:24:55.178246 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:55.181073 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 366/726] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 18 16:24:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:24:55.187212 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f76ea-925b-46d3-a513-e07c0bbe7763 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:55.204498 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b7c484-2d40-47c0-a231-e9a3f75d9afd None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:55.273248 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8db64e-89c4-4970-aa7e-13ceef471cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:55.274430 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8db64e-89c4-4970-aa7e-13ceef471cbf None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:55.274921 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8db64e-89c4-4970-aa7e-13ceef471cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:55.275399 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8db64e-89c4-4970-aa7e-13ceef471cbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:55.297337 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8db64e-89c4-4970-aa7e-13ceef471cbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:55.299157 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8db64e-89c4-4970-aa7e-13ceef471cbf None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:55.340957 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-cd8db64e-89c4-4970-aa7e-13ceef471cbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:55.340957 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:55.340957 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:55.340957 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:55.340957 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:55.340957 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:55.340957 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:55.340957 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:55.340957 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:55.340957 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:55.340957 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:55.340957 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:55.340957 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:55.340957 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:55.340957 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:55.341928 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:55.341928 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:55.341928 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:55.341928 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:55.341928 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:55.341928 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:55.341928 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:55.341928 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:55.341928 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:55.341928 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:55.341928 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:55.341928 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:55.341928 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:55.341928 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:55.341928 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:55.341928 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:55.341928 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:55.342851 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:55.342851 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:55.342851 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:55.342851 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:55.349233 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 361/727] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:55.356725 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8db64e-89c4-4970-aa7e-13ceef471cbf None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:55.369023 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f76ea-925b-46d3-a513-e07c0bbe7763 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:55.407314 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e410a703-8730-46e4-a04c-b648a1290af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:55.408401 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e410a703-8730-46e4-a04c-b648a1290af0 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:55.408891 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e410a703-8730-46e4-a04c-b648a1290af0 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:55.409412 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e410a703-8730-46e4-a04c-b648a1290af0 None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:55.410389 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e410a703-8730-46e4-a04c-b648a1290af0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:55.410939 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e410a703-8730-46e4-a04c-b648a1290af0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:55.412651 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e410a703-8730-46e4-a04c-b648a1290af0 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:55.423374 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 367/728] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 18 16:24:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:55.428395 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e410a703-8730-46e4-a04c-b648a1290af0 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:55.604460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8db64e-89c4-4970-aa7e-13ceef471cbf None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:55.661741 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5467cbf2-6a9d-47cb-a12a-934fa5c54435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:55.662411 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5467cbf2-6a9d-47cb-a12a-934fa5c54435 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:55.662694 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5467cbf2-6a9d-47cb-a12a-934fa5c54435 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:55.662969 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5467cbf2-6a9d-47cb-a12a-934fa5c54435 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:55.663728 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5467cbf2-6a9d-47cb-a12a-934fa5c54435 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:55.664035 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5467cbf2-6a9d-47cb-a12a-934fa5c54435 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:55.665708 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5467cbf2-6a9d-47cb-a12a-934fa5c54435 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:55.675904 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:55.680362 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 362/729] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:55.687404 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5467cbf2-6a9d-47cb-a12a-934fa5c54435 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:55.709775 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:55.765431 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e410a703-8730-46e4-a04c-b648a1290af0 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:55.802616 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c13225-34bb-4700-b06c-f4a2543e2aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:55.803420 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c13225-34bb-4700-b06c-f4a2543e2aa1 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:55.803726 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c13225-34bb-4700-b06c-f4a2543e2aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:55.804001 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c13225-34bb-4700-b06c-f4a2543e2aa1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:55.815571 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c13225-34bb-4700-b06c-f4a2543e2aa1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:55.817096 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c13225-34bb-4700-b06c-f4a2543e2aa1 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:55.836193 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-72c13225-34bb-4700-b06c-f4a2543e2aa1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:55.836193 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:55.836193 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:55.836193 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:55.836193 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:55.836193 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:55.836193 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:55.836193 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:55.836193 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:55.836193 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:55.836193 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:55.836193 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:55.836193 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:55.836193 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:55.836193 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:55.837139 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:55.837139 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:55.837139 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:55.837139 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:55.837139 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:55.837139 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:55.837139 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:55.837139 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:55.837139 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:55.837139 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:55.837139 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:55.837139 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:55.837139 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:55.837139 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:55.837139 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:55.837139 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:55.837139 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:55.838293 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:55.838293 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:55.838293 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:55.838293 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:55.838293 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 368/730] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 410 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:55.847331 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c13225-34bb-4700-b06c-f4a2543e2aa1 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:55.920195 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5467cbf2-6a9d-47cb-a12a-934fa5c54435 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:55.976048 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a1caf8-a3e1-4f63-b659-d817521e017b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:55.977653 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a1caf8-a3e1-4f63-b659-d817521e017b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:55.978543 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a1caf8-a3e1-4f63-b659-d817521e017b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:55.979025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a1caf8-a3e1-4f63-b659-d817521e017b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:55.990804 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a1caf8-a3e1-4f63-b659-d817521e017b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:55.992990 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a1caf8-a3e1-4f63-b659-d817521e017b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:56.005908 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c13225-34bb-4700-b06c-f4a2543e2aa1 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:56.013912 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:56.014495 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-64a1caf8-a3e1-4f63-b659-d817521e017b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 16:24:56.014495 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:56.014495 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:56.014495 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:56.014495 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:56.014495 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:56.014495 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:56.014495 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:56.014495 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:56.014495 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:56.014495 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:56.014495 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:56.014495 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:24:56.014495 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:24:56.014495 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:24:56.014495 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:24:56.015413 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:56.015413 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:56.015413 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:24:56.015413 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:56.015413 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:24:56.015413 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:56.015413 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:56.015413 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:56.015413 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:56.015413 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:56.015413 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:56.015413 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:56.015413 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:56.015413 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:56.015413 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:56.015413 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:56.015413 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:56.015413 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:56.016416 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:56.016416 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:56.016416 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:24:56.016416 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:24:56.016416 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:24:56.016416 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:24:56.016416 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:24:56.016416 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:24:56.016416 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 16:24:56.016416 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:56.021069 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 363/731] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 18 16:24:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:56.022775 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5183f998-35ce-4a91-b9a7-8a4f75788600 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:56.023240 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5183f998-35ce-4a91-b9a7-8a4f75788600 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:56.023678 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5183f998-35ce-4a91-b9a7-8a4f75788600 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:56.025437 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 364/732] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:56.029090 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5183f998-35ce-4a91-b9a7-8a4f75788600 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:56.058665 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89342ff7-9c16-4cd6-904c-b3fab1616fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:56.059637 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89342ff7-9c16-4cd6-904c-b3fab1616fb9 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:56.059925 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89342ff7-9c16-4cd6-904c-b3fab1616fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:56.060204 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89342ff7-9c16-4cd6-904c-b3fab1616fb9 None admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:56.060929 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89342ff7-9c16-4cd6-904c-b3fab1616fb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:56.061235 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89342ff7-9c16-4cd6-904c-b3fab1616fb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:56.063457 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89342ff7-9c16-4cd6-904c-b3fab1616fb9 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:56.088096 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 369/733] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:56.088096 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e92f8b-7c77-472f-827c-9868ac44434c None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:56.088096 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e92f8b-7c77-472f-827c-9868ac44434c None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:56.088096 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e92f8b-7c77-472f-827c-9868ac44434c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:56.108547 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-88e92f8b-7c77-472f-827c-9868ac44434c None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:56.109735 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-88e92f8b-7c77-472f-827c-9868ac44434c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:56.110239 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:56.161126 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5183f998-35ce-4a91-b9a7-8a4f75788600 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:56.187591 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 370/734] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:56.187591 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e92f8b-7c77-472f-827c-9868ac44434c None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:56.208999 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f636d2-dfbe-4231-92a1-2d75877d8212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:56.210129 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f636d2-dfbe-4231-92a1-2d75877d8212 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:56.210709 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f636d2-dfbe-4231-92a1-2d75877d8212 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:56.211201 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f636d2-dfbe-4231-92a1-2d75877d8212 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:56.212367 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f636d2-dfbe-4231-92a1-2d75877d8212 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:56.212872 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f636d2-dfbe-4231-92a1-2d75877d8212 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:56.216108 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f636d2-dfbe-4231-92a1-2d75877d8212 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:56.238555 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 365/735] 10.222.0.26 () {56 vars in 1135 bytes} [Mon Jul 18 16:24:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:56.239519 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f636d2-dfbe-4231-92a1-2d75877d8212 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:56.311141 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:56.319938 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e92f8b-7c77-472f-827c-9868ac44434c None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:56.360133 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a451fa64-2283-4edc-9e81-98d079fa7e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:56.361461 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a451fa64-2283-4edc-9e81-98d079fa7e99 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:56.362071 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a451fa64-2283-4edc-9e81-98d079fa7e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:56.362524 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a451fa64-2283-4edc-9e81-98d079fa7e99 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:56.365119 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f636d2-dfbe-4231-92a1-2d75877d8212 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:56.370837 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a451fa64-2283-4edc-9e81-98d079fa7e99 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:56.372592 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a451fa64-2283-4edc-9e81-98d079fa7e99 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:56.383438 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:56.390219 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-a451fa64-2283-4edc-9e81-98d079fa7e99 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 16:24:56.390219 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:56.390219 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:56.390219 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:56.390219 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:56.390219 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:56.390219 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:56.390219 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:56.390219 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:56.390219 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:56.390219 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:56.390219 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:56.390219 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:24:56.390219 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:24:56.390219 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:24:56.390219 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:24:56.391160 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:56.391160 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:56.391160 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:24:56.391160 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:56.391160 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:24:56.391160 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:56.391160 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:56.391160 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:56.391160 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:56.391160 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:56.391160 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:56.391160 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:56.391160 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:56.391160 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:56.391160 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:56.391160 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:56.391160 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:56.391160 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:56.392128 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:56.392128 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:56.392128 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:24:56.392128 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:24:56.392128 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:24:56.392128 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:24:56.392128 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:24:56.392128 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:24:56.392128 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 16:24:56.392128 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:56.394428 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 371/736] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Jul 18 16:24:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:56.395402 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a451fa64-2283-4edc-9e81-98d079fa7e99 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:56.409985 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f776ebf5-1451-49a3-a61b-16163410ddbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:56.410758 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f776ebf5-1451-49a3-a61b-16163410ddbd None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:56.411185 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f776ebf5-1451-49a3-a61b-16163410ddbd None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:56.411574 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f776ebf5-1451-49a3-a61b-16163410ddbd None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:56.412416 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f776ebf5-1451-49a3-a61b-16163410ddbd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:56.413849 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f776ebf5-1451-49a3-a61b-16163410ddbd None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:56.505686 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a451fa64-2283-4edc-9e81-98d079fa7e99 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:56.519960 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 366/737] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jul 18 16:24:56 2022] POST /identity/v3/services => generated 251 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:24:56.520907 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f776ebf5-1451-49a3-a61b-16163410ddbd None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:56.558126 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7dfccf-90b2-4640-a89d-076eab37671f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:56.559388 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7dfccf-90b2-4640-a89d-076eab37671f None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:56.560012 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7dfccf-90b2-4640-a89d-076eab37671f None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:56.560518 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7dfccf-90b2-4640-a89d-076eab37671f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:56.587328 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:56.643613 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7dfccf-90b2-4640-a89d-076eab37671f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:56.645662 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7dfccf-90b2-4640-a89d-076eab37671f None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:56.650506 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f776ebf5-1451-49a3-a61b-16163410ddbd None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:56.695767 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e55f2a6-0ef0-4c4a-9c1a-13cd293ac1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:56.696547 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e55f2a6-0ef0-4c4a-9c1a-13cd293ac1db None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:56.696834 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e55f2a6-0ef0-4c4a-9c1a-13cd293ac1db None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:56.697107 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e55f2a6-0ef0-4c4a-9c1a-13cd293ac1db None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:56.697945 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e55f2a6-0ef0-4c4a-9c1a-13cd293ac1db None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:56.698250 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e55f2a6-0ef0-4c4a-9c1a-13cd293ac1db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:56.700282 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e55f2a6-0ef0-4c4a-9c1a-13cd293ac1db None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:56.723667 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 367/738] 10.222.0.26 () {56 vars in 1149 bytes} [Mon Jul 18 16:24:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:56.724831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffb8af-e5b7-4e37-a81b-1742e71857cb None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:56.725110 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffb8af-e5b7-4e37-a81b-1742e71857cb None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:56.725398 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffb8af-e5b7-4e37-a81b-1742e71857cb None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:56.726782 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 368/739] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:56.733935 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ae78d-7a33-4499-9751-db2393754d71 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:56.734223 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ae78d-7a33-4499-9751-db2393754d71 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:56.734491 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ae78d-7a33-4499-9751-db2393754d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:56.754048 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-006ae78d-7a33-4499-9751-db2393754d71 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:56.755302 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-006ae78d-7a33-4499-9751-db2393754d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:56.778886 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 372/740] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:24:56 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:24:56.780425 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7dfccf-90b2-4640-a89d-076eab37671f None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:56.795539 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:56.838514 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 369/741] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:56 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:56.840792 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ae78d-7a33-4499-9751-db2393754d71 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:56.846421 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:56.908024 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7dfccf-90b2-4640-a89d-076eab37671f None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:56.943907 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef73cee3-2801-4ec2-9ab1-be33b0f8634d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:56.944810 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73cee3-2801-4ec2-9ab1-be33b0f8634d None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:56.945278 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73cee3-2801-4ec2-9ab1-be33b0f8634d None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:56.945557 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73cee3-2801-4ec2-9ab1-be33b0f8634d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:56.955121 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ae78d-7a33-4499-9751-db2393754d71 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:56.991440 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8135a30e-9888-46b8-b9ab-df1d0b289bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:56.992386 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8135a30e-9888-46b8-b9ab-df1d0b289bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:56.992667 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8135a30e-9888-46b8-b9ab-df1d0b289bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:56.992938 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8135a30e-9888-46b8-b9ab-df1d0b289bf7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:57.004962 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8135a30e-9888-46b8-b9ab-df1d0b289bf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:57.006937 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8135a30e-9888-46b8-b9ab-df1d0b289bf7 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:57.017633 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef73cee3-2801-4ec2-9ab1-be33b0f8634d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:57.019279 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef73cee3-2801-4ec2-9ab1-be33b0f8634d None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:57.026011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-8135a30e-9888-46b8-b9ab-df1d0b289bf7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:57.026011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:57.026011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:57.026011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:57.026011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:57.026011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:57.026011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:57.026011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:57.026011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:57.026011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:57.026011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:57.026011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:57.026011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:57.026011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:57.026011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:57.026991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:57.026991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:57.026991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:57.026991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:57.026991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:57.026991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:57.026991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:57.026991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:57.026991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:57.026991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:57.026991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:57.026991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:57.026991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:57.026991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:57.026991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:57.026991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:57.026991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:57.028116 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:57.028116 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:57.028116 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:57.028116 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:57.028116 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 370/742] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:57.029342 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a3986-f9b9-4f11-8033-db8a21a862a8 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:57.029342 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a3986-f9b9-4f11-8033-db8a21a862a8 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:57.029604 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a3986-f9b9-4f11-8033-db8a21a862a8 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:57.031103 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 371/743] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:57.032509 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841a3986-f9b9-4f11-8033-db8a21a862a8 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:57.053351 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:57.084725 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:57.140559 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 373/744] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:24:56 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:24:57.141221 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef73cee3-2801-4ec2-9ab1-be33b0f8634d None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:57.151073 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841a3986-f9b9-4f11-8033-db8a21a862a8 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:57.191920 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a098ea-6f41-4118-b06e-63b358f236bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:57.192709 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a098ea-6f41-4118-b06e-63b358f236bf None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:57.193035 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a098ea-6f41-4118-b06e-63b358f236bf None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:57.193347 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a098ea-6f41-4118-b06e-63b358f236bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:57.205477 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a098ea-6f41-4118-b06e-63b358f236bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:24:57.213192 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-45a098ea-6f41-4118-b06e-63b358f236bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:24:57.213192 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:57.213192 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:57.213192 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:57.213192 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:57.213192 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:57.213192 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:57.213192 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:57.213192 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:57.213192 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:57.213192 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:57.213192 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:57.213192 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:24:57.213192 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:24:57.213192 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:24:57.213192 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:24:57.214280 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:24:57.214280 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:24:57.214280 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:24:57.214280 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:24:57.214280 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:57.214280 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:57.214280 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:57.214280 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:57.214280 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:57.214280 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:57.214280 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:57.214280 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:57.214280 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:57.214280 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:57.214280 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:57.214280 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:57.216310 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:57.216310 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:57.216310 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:57.216310 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:57.216310 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:24:57.216310 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:24:57.216310 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:24:57.216310 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:24:57.216310 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:24:57.216310 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:24:57.216310 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:24:57.216310 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:57.216310 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 372/745] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 18 16:24:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:57.216310 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7131df-d1e4-429f-8658-486c9a4d9c6f None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:57.217110 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7131df-d1e4-429f-8658-486c9a4d9c6f None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:57.217110 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7131df-d1e4-429f-8658-486c9a4d9c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:57.235717 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-8b7131df-d1e4-429f-8658-486c9a4d9c6f None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:57.236925 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-8b7131df-d1e4-429f-8658-486c9a4d9c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:57.253002 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef73cee3-2801-4ec2-9ab1-be33b0f8634d None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:57.290564 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782f838b-4b01-4f0c-9e46-accca7aeb83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:57.291193 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f838b-4b01-4f0c-9e46-accca7aeb83e None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:57.291473 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f838b-4b01-4f0c-9e46-accca7aeb83e None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:57.291787 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f838b-4b01-4f0c-9e46-accca7aeb83e None admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:57.293103 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782f838b-4b01-4f0c-9e46-accca7aeb83e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:57.293648 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782f838b-4b01-4f0c-9e46-accca7aeb83e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:57.295107 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782f838b-4b01-4f0c-9e46-accca7aeb83e None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:57.313117 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 374/746] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:57.314027 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782f838b-4b01-4f0c-9e46-accca7aeb83e None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:57.315187 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 373/747] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:57 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:24:57.315948 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7131df-d1e4-429f-8658-486c9a4d9c6f None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:57.325025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:57.347699 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:57.426590 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7131df-d1e4-429f-8658-486c9a4d9c6f None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:57.442836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782f838b-4b01-4f0c-9e46-accca7aeb83e None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:57.465134 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ac581f-fcdb-4f42-9d52-3ec78d78d1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:57.465804 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac581f-fcdb-4f42-9d52-3ec78d78d1d6 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:57.466082 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac581f-fcdb-4f42-9d52-3ec78d78d1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:57.466354 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac581f-fcdb-4f42-9d52-3ec78d78d1d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:57.467099 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ac581f-fcdb-4f42-9d52-3ec78d78d1d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:57.467403 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ac581f-fcdb-4f42-9d52-3ec78d78d1d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:57.468820 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ac581f-fcdb-4f42-9d52-3ec78d78d1d6 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:57.478143 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c16f90-7212-4597-bf9e-3bf68ed34f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:57.479014 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c16f90-7212-4597-bf9e-3bf68ed34f13 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:57.479297 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c16f90-7212-4597-bf9e-3bf68ed34f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:57.479604 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c16f90-7212-4597-bf9e-3bf68ed34f13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:57.481889 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 374/748] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:24:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:57.482764 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ac581f-fcdb-4f42-9d52-3ec78d78d1d6 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:57.490904 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c16f90-7212-4597-bf9e-3bf68ed34f13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:57.492385 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c16f90-7212-4597-bf9e-3bf68ed34f13 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:57.511055 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-06c16f90-7212-4597-bf9e-3bf68ed34f13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:57.511055 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:57.511055 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:57.511055 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:57.511055 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:57.511055 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:57.511055 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:57.511055 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:57.511055 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:57.511055 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:57.511055 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:57.511055 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:57.511055 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:57.511055 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:57.511055 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:57.512669 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:57.512669 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:57.512669 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:57.512669 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:57.512669 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:57.512669 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:57.512669 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:57.512669 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:57.512669 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:57.512669 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:57.512669 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:57.512669 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:57.512669 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:57.512669 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:57.512669 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:57.512669 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:57.512669 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:57.513645 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:57.513645 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:57.513645 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:57.513645 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:57.513645 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 375/749] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:57.514311 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c16f90-7212-4597-bf9e-3bf68ed34f13 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:57.563172 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:57.578877 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:57.600987 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ac581f-fcdb-4f42-9d52-3ec78d78d1d6 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:57.623698 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c16f90-7212-4597-bf9e-3bf68ed34f13 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:57.638044 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2181a690-c5f8-4819-a067-204814b852c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:57.638850 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2181a690-c5f8-4819-a067-204814b852c3 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:57.639132 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2181a690-c5f8-4819-a067-204814b852c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:57.639414 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2181a690-c5f8-4819-a067-204814b852c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:57.646730 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2181a690-c5f8-4819-a067-204814b852c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:57.648214 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2181a690-c5f8-4819-a067-204814b852c3 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:57.661184 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df798c17-f081-4431-8636-f8986bf175ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:57.661852 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df798c17-f081-4431-8636-f8986bf175ce None admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:57.662135 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df798c17-f081-4431-8636-f8986bf175ce None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:57.662410 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df798c17-f081-4431-8636-f8986bf175ce None admin] PATH_INFO: `/v3/services` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:57.663126 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df798c17-f081-4431-8636-f8986bf175ce None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:57.664132 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-2181a690-c5f8-4819-a067-204814b852c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:57.664132 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:57.664132 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:57.664132 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:57.664132 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:57.664132 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:57.664132 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:57.664132 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:57.664132 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:57.664132 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:57.664132 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:57.664132 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:57.664132 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:24:57.664132 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:24:57.664132 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:24:57.664992 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:24:57.664992 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:57.664992 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:57.664992 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:57.664992 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:57.664992 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:57.664992 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:57.664992 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:57.664992 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:57.664992 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:57.664992 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:57.664992 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:57.664992 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:57.664992 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:57.664992 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:57.664992 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:57.664992 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:57.665901 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:24:57.665901 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:24:57.665901 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:24:57.665901 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:24:57.665901 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:24:57.665901 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:24:57.665901 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:57.665901 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:57.666339 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df798c17-f081-4431-8636-f8986bf175ce None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:57.671170 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 375/750] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:24:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:57.672121 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2181a690-c5f8-4819-a067-204814b852c3 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:57.741667 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 376/751] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jul 18 16:24:57 2022] POST /identity/v3/services => generated 240 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:24:57.742327 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df798c17-f081-4431-8636-f8986bf175ce None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:57.804641 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2181a690-c5f8-4819-a067-204814b852c3 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:57.854147 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:57.884506 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca251d2a-bd8b-4639-9263-a5a8d125da1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:57.885622 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca251d2a-bd8b-4639-9263-a5a8d125da1c None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:57.886482 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca251d2a-bd8b-4639-9263-a5a8d125da1c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:57.886778 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca251d2a-bd8b-4639-9263-a5a8d125da1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:57.888233 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df798c17-f081-4431-8636-f8986bf175ce None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:57.900660 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca251d2a-bd8b-4639-9263-a5a8d125da1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:57.903359 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca251d2a-bd8b-4639-9263-a5a8d125da1c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:57.915942 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:57.926610 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-ca251d2a-bd8b-4639-9263-a5a8d125da1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:57.926610 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:57.926610 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:57.926610 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:57.926610 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:57.926610 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:57.926610 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:57.926610 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:57.926610 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:57.926610 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:57.926610 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:57.926610 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:57.926610 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:57.926610 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:57.926610 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:57.928987 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:57.928987 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:57.928987 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:57.928987 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:57.928987 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:57.928987 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:57.928987 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:57.928987 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:57.928987 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:57.928987 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:57.928987 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:57.928987 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:57.928987 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:57.928987 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:57.928987 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:57.928987 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:57.928987 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:57.929865 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:57.929865 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:57.929865 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:57.929865 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:57.929865 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 376/752] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:57.929865 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca251d2a-bd8b-4639-9263-a5a8d125da1c None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:57.933641 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60648c73-b112-44f7-ab8e-34213f8f21c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:57.935187 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60648c73-b112-44f7-ab8e-34213f8f21c8 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:57.935479 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60648c73-b112-44f7-ab8e-34213f8f21c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:57.935823 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60648c73-b112-44f7-ab8e-34213f8f21c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:57.936569 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60648c73-b112-44f7-ab8e-34213f8f21c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:57.936881 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60648c73-b112-44f7-ab8e-34213f8f21c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:57.938272 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60648c73-b112-44f7-ab8e-34213f8f21c8 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:57.954870 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 377/753] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:57.956080 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60648c73-b112-44f7-ab8e-34213f8f21c8 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:58.058781 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca251d2a-bd8b-4639-9263-a5a8d125da1c None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:58.095735 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055efb5a-aace-4d64-98b6-7bd10466d8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:58.096323 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055efb5a-aace-4d64-98b6-7bd10466d8ab None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:58.096605 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055efb5a-aace-4d64-98b6-7bd10466d8ab None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:58.096917 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055efb5a-aace-4d64-98b6-7bd10466d8ab None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:58.098492 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055efb5a-aace-4d64-98b6-7bd10466d8ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:58.098836 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055efb5a-aace-4d64-98b6-7bd10466d8ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:58.100721 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055efb5a-aace-4d64-98b6-7bd10466d8ab None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:58.101890 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60648c73-b112-44f7-ab8e-34213f8f21c8 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:58.119525 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 377/754] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 18 16:24:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:58.119525 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055efb5a-aace-4d64-98b6-7bd10466d8ab None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:58.154661 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfe18ee-3949-409d-8a1f-d0f1a57ad3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:58.155979 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe18ee-3949-409d-8a1f-d0f1a57ad3b9 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:58.156452 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe18ee-3949-409d-8a1f-d0f1a57ad3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:58.157046 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe18ee-3949-409d-8a1f-d0f1a57ad3b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:58.158463 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfe18ee-3949-409d-8a1f-d0f1a57ad3b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:58.159080 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfe18ee-3949-409d-8a1f-d0f1a57ad3b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:58.161193 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfe18ee-3949-409d-8a1f-d0f1a57ad3b9 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:58.164816 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:58.179132 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 378/755] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:58.180584 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfe18ee-3949-409d-8a1f-d0f1a57ad3b9 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:58.236998 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055efb5a-aace-4d64-98b6-7bd10466d8ab None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:58.237954 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:58.282579 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32012548-b3b7-480c-b5c0-9310ade3402d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:58.283700 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32012548-b3b7-480c-b5c0-9310ade3402d None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:58.283816 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32012548-b3b7-480c-b5c0-9310ade3402d None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:58.284140 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32012548-b3b7-480c-b5c0-9310ade3402d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:58.291473 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32012548-b3b7-480c-b5c0-9310ade3402d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:58.294080 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32012548-b3b7-480c-b5c0-9310ade3402d None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:58.309311 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-32012548-b3b7-480c-b5c0-9310ade3402d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:58.309311 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:58.309311 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:58.309311 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:58.309311 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:58.309311 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:58.309311 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:58.309311 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:58.309311 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:58.309311 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:58.309311 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:58.309311 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:58.309311 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:24:58.309311 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:24:58.309311 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:24:58.310275 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:24:58.310275 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:58.310275 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:58.310275 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:58.310275 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:58.310275 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:58.310275 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:58.310275 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:58.310275 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:58.310275 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:58.310275 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:58.310275 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:58.310275 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:58.310275 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:58.310275 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:58.310275 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:58.310275 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:58.311177 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:24:58.311177 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:24:58.311177 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:24:58.311177 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:24:58.311177 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:24:58.311177 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:24:58.311177 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:58.311177 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:58.311177 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 378/756] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:24:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:58.312455 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32012548-b3b7-480c-b5c0-9310ade3402d None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:58.316646 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfe18ee-3949-409d-8a1f-d0f1a57ad3b9 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:58.358930 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1571929e-bb33-4495-8ef7-bbf8ccd41a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:58.359703 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571929e-bb33-4495-8ef7-bbf8ccd41a85 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:58.360087 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571929e-bb33-4495-8ef7-bbf8ccd41a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:58.360463 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571929e-bb33-4495-8ef7-bbf8ccd41a85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:58.361411 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1571929e-bb33-4495-8ef7-bbf8ccd41a85 None admin] RBAC: Adding query filter params (role.id=98eaadb3669f4b608d572c001742f559, scope.project.id=eb9747e2299c4e0991fd7edb4462c7e5, user.id=43854d7a6df2466eb7976fabe6011767) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:58.361840 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1571929e-bb33-4495-8ef7-bbf8ccd41a85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:58.363380 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1571929e-bb33-4495-8ef7-bbf8ccd41a85 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:58.375681 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 379/757] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 18 16:24:58 2022] GET /identity/v3/role_assignments?user.id=43854d7a6df2466eb7976fabe6011767&scope.project.id=eb9747e2299c4e0991fd7edb4462c7e5&role.id=98eaadb3669f4b608d572c001742f559 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:58.376821 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1571929e-bb33-4495-8ef7-bbf8ccd41a85 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:58.437565 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:58.446651 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32012548-b3b7-480c-b5c0-9310ade3402d None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:58.496192 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5c15e-94ce-4058-b7c1-18b79ea0eb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:58.498528 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5c15e-94ce-4058-b7c1-18b79ea0eb98 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:58.499855 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5c15e-94ce-4058-b7c1-18b79ea0eb98 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:58.500324 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5c15e-94ce-4058-b7c1-18b79ea0eb98 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:58.520270 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f5c15e-94ce-4058-b7c1-18b79ea0eb98 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:58.524405 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f5c15e-94ce-4058-b7c1-18b79ea0eb98 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:58.561021 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-06f5c15e-94ce-4058-b7c1-18b79ea0eb98 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:24:58.561021 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:58.561021 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:58.561021 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:58.561021 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:58.561021 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:58.561021 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:58.561021 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:58.561021 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:58.561021 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:58.561021 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:58.561021 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:58.561021 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:24:58.561021 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:24:58.561021 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:24:58.561021 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:24:58.563204 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:58.563204 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:58.563204 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:24:58.563204 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:58.563204 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:24:58.563204 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:24:58.563204 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:58.563204 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:58.563204 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:58.563204 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:58.563204 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:58.563204 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:58.563204 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:58.563204 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:58.563204 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:58.563204 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:58.563204 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:58.563204 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:58.565339 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:58.565339 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:58.565339 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:24:58.565339 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:24:58.565339 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:24:58.565339 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:24:58.565339 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:24:58.565339 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:24:58.565339 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:24:58.565339 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:58.575431 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 379/758] 10.222.0.26 () {56 vars in 1093 bytes} [Mon Jul 18 16:24:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:58.585460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5c15e-94ce-4058-b7c1-18b79ea0eb98 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:58.607274 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1571929e-bb33-4495-8ef7-bbf8ccd41a85 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:58.611046 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:58.655308 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28151479-09c1-4749-9bf8-8dfdf85cc65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:58.656495 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28151479-09c1-4749-9bf8-8dfdf85cc65a None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:58.656816 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28151479-09c1-4749-9bf8-8dfdf85cc65a None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:58.657135 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28151479-09c1-4749-9bf8-8dfdf85cc65a None admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:58.657973 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28151479-09c1-4749-9bf8-8dfdf85cc65a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:58.658288 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28151479-09c1-4749-9bf8-8dfdf85cc65a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:58.662447 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28151479-09c1-4749-9bf8-8dfdf85cc65a None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:58.686882 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 380/759] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 18 16:24:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:58.690972 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974084df-8f70-4ddd-b2e4-f24920d852fc None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:58.691260 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974084df-8f70-4ddd-b2e4-f24920d852fc None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:58.691563 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974084df-8f70-4ddd-b2e4-f24920d852fc None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:58.692995 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 381/760] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:24:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:24:58.693619 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974084df-8f70-4ddd-b2e4-f24920d852fc None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:58.740101 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5c15e-94ce-4058-b7c1-18b79ea0eb98 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:58.779186 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c8f68c-07df-44f9-a149-f49e0bb62f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:58.780248 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8f68c-07df-44f9-a149-f49e0bb62f81 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:58.780739 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8f68c-07df-44f9-a149-f49e0bb62f81 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:58.781232 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8f68c-07df-44f9-a149-f49e0bb62f81 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:58.782272 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c8f68c-07df-44f9-a149-f49e0bb62f81 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:58.782801 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c8f68c-07df-44f9-a149-f49e0bb62f81 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:58.784487 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c8f68c-07df-44f9-a149-f49e0bb62f81 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:58.809760 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 380/761] 10.222.0.26 () {56 vars in 1133 bytes} [Mon Jul 18 16:24:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:58.811535 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64298764-3374-4710-bb7b-4435494d092f None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:58.812031 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64298764-3374-4710-bb7b-4435494d092f None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:58.812534 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64298764-3374-4710-bb7b-4435494d092f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:58.824507 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974084df-8f70-4ddd-b2e4-f24920d852fc None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:58.831059 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-64298764-3374-4710-bb7b-4435494d092f None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:24:58.832573 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-64298764-3374-4710-bb7b-4435494d092f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:24:58.843833 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:58.862725 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0844563-41cf-4ef7-b69d-5e9f86fad96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:58.863346 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0844563-41cf-4ef7-b69d-5e9f86fad96b None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:58.863657 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0844563-41cf-4ef7-b69d-5e9f86fad96b None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:58.863938 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0844563-41cf-4ef7-b69d-5e9f86fad96b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:58.864824 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0844563-41cf-4ef7-b69d-5e9f86fad96b None admin] RBAC: Adding query filter params (role.id=4fe970e98f394bbfb096c4879bebdf97, scope.project.id=eb9747e2299c4e0991fd7edb4462c7e5, user.id=da799b0044f54da6835aa06e43ce1a6f) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:58.865137 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0844563-41cf-4ef7-b69d-5e9f86fad96b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:58.866614 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0844563-41cf-4ef7-b69d-5e9f86fad96b None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:58.873310 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 382/762] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 18 16:24:58 2022] GET /identity/v3/role_assignments?user.id=da799b0044f54da6835aa06e43ce1a6f&scope.project.id=eb9747e2299c4e0991fd7edb4462c7e5&role.id=4fe970e98f394bbfb096c4879bebdf97 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:58.874913 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0844563-41cf-4ef7-b69d-5e9f86fad96b None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:58.913613 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 381/763] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:24:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:24:58.938632 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:58.939596 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64298764-3374-4710-bb7b-4435494d092f None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:59.057485 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64298764-3374-4710-bb7b-4435494d092f None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:59.063000 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0844563-41cf-4ef7-b69d-5e9f86fad96b None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:59.096150 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdda613-52dd-4dca-aef2-5e21a89c4121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:59.097744 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdda613-52dd-4dca-aef2-5e21a89c4121 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:59.098243 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdda613-52dd-4dca-aef2-5e21a89c4121 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:59.098720 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdda613-52dd-4dca-aef2-5e21a89c4121 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:59.099651 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdda613-52dd-4dca-aef2-5e21a89c4121 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:59.101277 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdda613-52dd-4dca-aef2-5e21a89c4121 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:59.103913 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1545436f-dcec-4e09-9be4-e6f9fc037c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:59.105297 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1545436f-dcec-4e09-9be4-e6f9fc037c95 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:59.105573 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1545436f-dcec-4e09-9be4-e6f9fc037c95 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:59.105848 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1545436f-dcec-4e09-9be4-e6f9fc037c95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:59.117518 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1545436f-dcec-4e09-9be4-e6f9fc037c95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:59.119038 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1545436f-dcec-4e09-9be4-e6f9fc037c95 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:59.120625 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-efdda613-52dd-4dca-aef2-5e21a89c4121 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 16:24:59.120625 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:59.120625 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:24:59.120625 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:24:59.120625 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:24:59.120625 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:24:59.120625 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:24:59.120625 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:24:59.120625 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 16:24:59.120625 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:59.120625 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:24:59.120625 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:59.120625 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:59.120625 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:59.120625 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:59.120625 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:59.121806 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:59.121806 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:59.121806 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:59.121806 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:59.121806 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:59.121806 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:59.121806 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:59.121806 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:24:59.121806 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:24:59.121806 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:24:59.121806 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:24:59.121806 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:59.121806 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:59.121806 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:59.121806 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:59.121806 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:59.122662 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:59.122662 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:59.122662 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:59.122662 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:59.122662 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:59.122662 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:59.122662 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:59.122662 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:59.122662 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:59.122662 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:59.122662 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:59.122662 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:24:59.122662 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:24:59.122662 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 16:24:59.122662 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:24:59.125780 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 382/764] 10.222.0.26 () {56 vars in 1129 bytes} [Mon Jul 18 16:24:58 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:24:59.131900 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdda613-52dd-4dca-aef2-5e21a89c4121 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:59.139654 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-1545436f-dcec-4e09-9be4-e6f9fc037c95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:59.139654 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:59.139654 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:59.139654 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:59.139654 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:59.139654 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:59.139654 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:59.139654 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:59.139654 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:59.139654 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:59.139654 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:59.139654 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:59.139654 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:24:59.139654 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:24:59.139654 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:24:59.140648 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:24:59.140648 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:59.140648 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:59.140648 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:59.140648 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:59.140648 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:59.140648 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:59.140648 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:59.140648 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:59.140648 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:59.140648 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:59.140648 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:59.140648 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:59.140648 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:59.140648 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:59.140648 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:59.140648 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:59.141543 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:24:59.141543 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:24:59.141543 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:24:59.141543 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:59.147471 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:59.148762 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 383/765] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:24:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:24:59.156332 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1545436f-dcec-4e09-9be4-e6f9fc037c95 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:59.320707 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdda613-52dd-4dca-aef2-5e21a89c4121 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:59.331789 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:59.355252 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1545436f-dcec-4e09-9be4-e6f9fc037c95 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:59.392754 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b901d4b7-d516-49bf-b02d-9dcf82b9eecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:59.393355 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901d4b7-d516-49bf-b02d-9dcf82b9eecd None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:59.393642 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901d4b7-d516-49bf-b02d-9dcf82b9eecd None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:59.393916 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901d4b7-d516-49bf-b02d-9dcf82b9eecd None admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:59.394634 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b901d4b7-d516-49bf-b02d-9dcf82b9eecd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:59.394941 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b901d4b7-d516-49bf-b02d-9dcf82b9eecd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:59.396329 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b901d4b7-d516-49bf-b02d-9dcf82b9eecd None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:59.410196 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 384/766] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:24:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:59.412801 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0512f3d2-1344-42e8-ba72-76b67d7b7464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:59.413486 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512f3d2-1344-42e8-ba72-76b67d7b7464 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:59.413766 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512f3d2-1344-42e8-ba72-76b67d7b7464 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:59.414040 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512f3d2-1344-42e8-ba72-76b67d7b7464 None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:59.414754 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0512f3d2-1344-42e8-ba72-76b67d7b7464 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:59.415059 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0512f3d2-1344-42e8-ba72-76b67d7b7464 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:59.416921 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b901d4b7-d516-49bf-b02d-9dcf82b9eecd None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:59.424383 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0512f3d2-1344-42e8-ba72-76b67d7b7464 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:59.432482 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 383/767] 10.222.0.26 () {56 vars in 1124 bytes} [Mon Jul 18 16:24:59 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:59.438830 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0512f3d2-1344-42e8-ba72-76b67d7b7464 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:59.549376 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b901d4b7-d516-49bf-b02d-9dcf82b9eecd None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:59.585715 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa0069-9bf4-4af1-ad41-acb362e7de22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:59.599321 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa0069-9bf4-4af1-ad41-acb362e7de22 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:59.599321 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa0069-9bf4-4af1-ad41-acb362e7de22 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:59.599321 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa0069-9bf4-4af1-ad41-acb362e7de22 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:59.599321 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aa0069-9bf4-4af1-ad41-acb362e7de22 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:59.599321 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aa0069-9bf4-4af1-ad41-acb362e7de22 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:59.599321 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:59.612533 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-88aa0069-9bf4-4af1-ad41-acb362e7de22 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:59.612533 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:24:59.612533 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:24:59.612533 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:24:59.612533 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:24:59.612533 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:24:59.612533 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:24:59.612533 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:24:59.612533 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:24:59.612533 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:24:59.612533 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:24:59.612533 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:24:59.612533 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:24:59.612533 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:24:59.612533 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:24:59.613461 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:24:59.613461 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:24:59.613461 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:24:59.613461 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:24:59.613461 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:24:59.613461 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:24:59.613461 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:24:59.613461 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:24:59.613461 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:24:59.613461 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:24:59.613461 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:24:59.613461 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:24:59.613461 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:24:59.613461 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:24:59.613461 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:24:59.613461 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:24:59.613461 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:24:59.614343 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:24:59.614343 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:24:59.614343 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:24:59.614343 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:24:59.614343 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:24:59.614343 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:24:59.614343 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:24:59.614343 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:24:59.614343 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 385/768] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:24:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:24:59.622473 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa0069-9bf4-4af1-ad41-acb362e7de22 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:59.645650 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:24:59.737842 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0512f3d2-1344-42e8-ba72-76b67d7b7464 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:59.752233 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa0069-9bf4-4af1-ad41-acb362e7de22 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:24:59.789100 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cceca38-5f15-4441-9cd3-ea4dd6ea5bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:59.789774 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cceca38-5f15-4441-9cd3-ea4dd6ea5bf1 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:59.790054 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cceca38-5f15-4441-9cd3-ea4dd6ea5bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:59.790329 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cceca38-5f15-4441-9cd3-ea4dd6ea5bf1 None admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:59.791121 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cceca38-5f15-4441-9cd3-ea4dd6ea5bf1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:59.791425 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cceca38-5f15-4441-9cd3-ea4dd6ea5bf1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:59.794391 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cceca38-5f15-4441-9cd3-ea4dd6ea5bf1 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:59.814793 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 386/769] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 18 16:24:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:24:59.821507 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cceca38-5f15-4441-9cd3-ea4dd6ea5bf1 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:59.841568 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd606d9-9ee9-45c0-82bb-02ad267eae0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:24:59.842740 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd606d9-9ee9-45c0-82bb-02ad267eae0a None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:24:59.843315 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd606d9-9ee9-45c0-82bb-02ad267eae0a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:24:59.843902 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd606d9-9ee9-45c0-82bb-02ad267eae0a None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:24:59.844885 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd606d9-9ee9-45c0-82bb-02ad267eae0a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:24:59.845540 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd606d9-9ee9-45c0-82bb-02ad267eae0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:24:59.848538 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd606d9-9ee9-45c0-82bb-02ad267eae0a None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:24:59.873838 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 384/770] 10.222.0.26 () {56 vars in 1124 bytes} [Mon Jul 18 16:24:59 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 435 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:24:59.881308 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd606d9-9ee9-45c0-82bb-02ad267eae0a None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:24:59.933103 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:00.014366 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cceca38-5f15-4441-9cd3-ea4dd6ea5bf1 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:00.076477 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd606d9-9ee9-45c0-82bb-02ad267eae0a None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:00.085153 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96a579c-eabb-4c4c-99e8-e0dc6783b2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:00.086067 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a579c-eabb-4c4c-99e8-e0dc6783b2e8 None admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:00.086451 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a579c-eabb-4c4c-99e8-e0dc6783b2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:00.086735 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a579c-eabb-4c4c-99e8-e0dc6783b2e8 None admin] PATH_INFO: `/v3/projects/eb9747e2299c4e0991fd7edb4462c7e5/users/5b868bfa44b94ca5b470166d39f20e54/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:00.089390 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96a579c-eabb-4c4c-99e8-e0dc6783b2e8 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb9747e2299c4e0991fd7edb4462c7e5, user_id=5b868bfa44b94ca5b470166d39f20e54, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:00.090964 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96a579c-eabb-4c4c-99e8-e0dc6783b2e8 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:00.115082 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbda1d0-78c6-4335-9073-ee83eb89a502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:00.115827 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbda1d0-78c6-4335-9073-ee83eb89a502 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:00.116150 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbda1d0-78c6-4335-9073-ee83eb89a502 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:00.116435 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbda1d0-78c6-4335-9073-ee83eb89a502 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:00.117227 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbda1d0-78c6-4335-9073-ee83eb89a502 None admin] RBAC: Adding query filter params (interface=public, service_id=3370792b52cf40fa938ed3b79d04ba29, region_id=RegionOne) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:00.117605 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbda1d0-78c6-4335-9073-ee83eb89a502 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:00.118986 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbda1d0-78c6-4335-9073-ee83eb89a502 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:00.124631 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:00.128074 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 385/771] 10.222.0.26 () {56 vars in 1251 bytes} [Mon Jul 18 16:24:59 2022] GET /identity/v3/endpoints?service_id=3370792b52cf40fa938ed3b79d04ba29&interface=public®ion_id=RegionOne => generated 194 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:00.129197 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c510e-07d1-4044-97b6-9c0dd3e0c6ad None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:00.129474 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c510e-07d1-4044-97b6-9c0dd3e0c6ad None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:00.129752 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c510e-07d1-4044-97b6-9c0dd3e0c6ad None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:00.131117 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 386/772] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:25:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:00.137469 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4329f85-ead3-4cb6-825c-461616314a06 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:00.137739 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4329f85-ead3-4cb6-825c-461616314a06 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:00.138019 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4329f85-ead3-4cb6-825c-461616314a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:00.155358 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-c4329f85-ead3-4cb6-825c-461616314a06 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:00.156627 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-c4329f85-ead3-4cb6-825c-461616314a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:00.190583 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 387/773] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 18 16:24:59 2022] PUT /identity/v3/projects/eb9747e2299c4e0991fd7edb4462c7e5/users/5b868bfa44b94ca5b470166d39f20e54/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:25:00.190583 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96a579c-eabb-4c4c-99e8-e0dc6783b2e8 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:00.296006 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 387/774] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:00.325708 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4329f85-ead3-4cb6-825c-461616314a06 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:00.406777 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96a579c-eabb-4c4c-99e8-e0dc6783b2e8 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:00.442229 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:00.447526 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7251cdbe-02af-419b-8954-c95a6d5d9d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:00.449836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7251cdbe-02af-419b-8954-c95a6d5d9d6f None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:00.450120 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7251cdbe-02af-419b-8954-c95a6d5d9d6f None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:00.450397 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7251cdbe-02af-419b-8954-c95a6d5d9d6f None admin] PATH_INFO: `/v3/services` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:00.463547 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7251cdbe-02af-419b-8954-c95a6d5d9d6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:00.463547 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7251cdbe-02af-419b-8954-c95a6d5d9d6f None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:00.463547 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 388/775] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Jul 18 16:25:00 2022] GET /identity/v3/services => generated 772 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:00.496333 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4329f85-ead3-4cb6-825c-461616314a06 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:00.532605 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:00.539282 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d142e-2277-4bb0-9d16-028558c8d164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:00.540581 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d142e-2277-4bb0-9d16-028558c8d164 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:00.541046 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d142e-2277-4bb0-9d16-028558c8d164 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:00.543609 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d142e-2277-4bb0-9d16-028558c8d164 None admin] PATH_INFO: `/v3/services/network` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:00.543609 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d142e-2277-4bb0-9d16-028558c8d164 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:00.544258 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d142e-2277-4bb0-9d16-028558c8d164 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:00.558144 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-407d142e-2277-4bb0-9d16-028558c8d164 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 16:25:00.558144 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:00.558144 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:25:00.558144 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:25:00.558144 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:25:00.558144 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:25:00.558144 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:25:00.558144 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:00.558144 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 16:25:00.558144 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:00.558144 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:25:00.558144 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:00.558144 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:00.558144 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:00.558144 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:00.558144 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:00.558144 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:00.559200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:00.559200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:00.559200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:00.559200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:00.559200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:00.559200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:00.559200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:25:00.559200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:25:00.559200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:25:00.559200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:25:00.559200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:00.559200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:00.559200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:00.559200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:00.559200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:00.559200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:00.560105 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:00.560105 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:00.560105 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:00.560105 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:00.560105 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:00.560105 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:00.560105 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:00.560105 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:00.560105 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:00.560105 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:00.560105 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:25:00.560105 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:00.560105 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 16:25:00.560105 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:00.564036 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 388/776] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:25:00 2022] GET /identity/v3/services/network => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:25:00.575526 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7251cdbe-02af-419b-8954-c95a6d5d9d6f None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:00.736696 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7251cdbe-02af-419b-8954-c95a6d5d9d6f None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:00.744587 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a180000-1555-478b-afdd-7b8a0396a2f4 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:00.745914 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a180000-1555-478b-afdd-7b8a0396a2f4 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:00.746408 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a180000-1555-478b-afdd-7b8a0396a2f4 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:00.748419 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 389/777] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:25:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:00.754137 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce9604-1905-4410-bfd7-b71091f4ca2c None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:00.755088 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce9604-1905-4410-bfd7-b71091f4ca2c None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:00.755612 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce9604-1905-4410-bfd7-b71091f4ca2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:00.780881 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-02ce9604-1905-4410-bfd7-b71091f4ca2c None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:00.783037 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-02ce9604-1905-4410-bfd7-b71091f4ca2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:00.786203 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697fd476-7527-4b94-9077-373176f833fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:00.787433 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697fd476-7527-4b94-9077-373176f833fd None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:00.787433 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697fd476-7527-4b94-9077-373176f833fd None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:00.787433 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697fd476-7527-4b94-9077-373176f833fd None admin] PATH_INFO: `/v3/services` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:00.788980 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697fd476-7527-4b94-9077-373176f833fd None admin] RBAC: Adding query filter params (name=network) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:00.789290 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697fd476-7527-4b94-9077-373176f833fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:00.791375 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697fd476-7527-4b94-9077-373176f833fd None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:00.796630 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 389/778] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Jul 18 16:25:00 2022] GET /identity/v3/services?name=network => generated 124 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:00.802288 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697fd476-7527-4b94-9077-373176f833fd None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:00.879965 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 390/779] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:00.887889 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:00.906024 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce9604-1905-4410-bfd7-b71091f4ca2c None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:00.942675 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697fd476-7527-4b94-9077-373176f833fd None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:00.981985 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b77d740-0689-4599-a072-3347b19a7e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:00.982667 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b77d740-0689-4599-a072-3347b19a7e70 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:00.982955 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b77d740-0689-4599-a072-3347b19a7e70 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:00.983234 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b77d740-0689-4599-a072-3347b19a7e70 None admin] PATH_INFO: `/v3/services` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:00.984011 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b77d740-0689-4599-a072-3347b19a7e70 None admin] RBAC: Adding query filter params (type=network) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:00.984327 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b77d740-0689-4599-a072-3347b19a7e70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:00.985856 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b77d740-0689-4599-a072-3347b19a7e70 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:00.992146 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 390/780] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Jul 18 16:25:00 2022] GET /identity/v3/services?type=network => generated 350 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:00.998655 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b77d740-0689-4599-a072-3347b19a7e70 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:01.112591 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b77d740-0689-4599-a072-3347b19a7e70 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:01.143962 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:01.155847 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d488f54d-14fa-4161-93dd-47179a9fa9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:01.156544 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d488f54d-14fa-4161-93dd-47179a9fa9fc None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:01.157018 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d488f54d-14fa-4161-93dd-47179a9fa9fc None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:01.157421 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d488f54d-14fa-4161-93dd-47179a9fa9fc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:01.158252 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d488f54d-14fa-4161-93dd-47179a9fa9fc None admin] RBAC: Adding query filter params (interface=public, service_id=3c13e151234d4db78d670826ec6d0699, region_id=RegionOne) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:01.158574 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d488f54d-14fa-4161-93dd-47179a9fa9fc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:01.160304 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d488f54d-14fa-4161-93dd-47179a9fa9fc None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:01.169409 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 391/781] 10.222.0.26 () {56 vars in 1251 bytes} [Mon Jul 18 16:25:00 2022] GET /identity/v3/endpoints?service_id=3c13e151234d4db78d670826ec6d0699&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:01.175700 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d488f54d-14fa-4161-93dd-47179a9fa9fc None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:01.221748 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce9604-1905-4410-bfd7-b71091f4ca2c None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:01.296947 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:01.299371 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d488f54d-14fa-4161-93dd-47179a9fa9fc None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:01.317520 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fcb9fb-98bb-4901-af86-1d3668a11064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:01.318874 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fcb9fb-98bb-4901-af86-1d3668a11064 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:01.319305 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fcb9fb-98bb-4901-af86-1d3668a11064 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:01.319937 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fcb9fb-98bb-4901-af86-1d3668a11064 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:01.337127 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fcb9fb-98bb-4901-af86-1d3668a11064 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:25:01.339549 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afbb5a2-6b90-4f78-98ea-31d3638d4890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:01.340348 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afbb5a2-6b90-4f78-98ea-31d3638d4890 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:01.340727 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afbb5a2-6b90-4f78-98ea-31d3638d4890 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:01.341101 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afbb5a2-6b90-4f78-98ea-31d3638d4890 None admin] PATH_INFO: `/v3/services` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:01.341926 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afbb5a2-6b90-4f78-98ea-31d3638d4890 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:01.343388 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afbb5a2-6b90-4f78-98ea-31d3638d4890 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:01.347714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-98fcb9fb-98bb-4901-af86-1d3668a11064 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:25:01.347714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:01.347714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:01.347714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:01.347714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:01.347714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:01.347714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:01.347714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:01.347714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:01.347714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:01.347714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:01.347714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:01.347714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:25:01.347714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:25:01.347714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:25:01.347714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:25:01.348723 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:25:01.348723 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:25:01.348723 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:25:01.348723 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:25:01.348723 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:01.348723 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:01.348723 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:01.348723 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:01.348723 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:01.348723 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:01.348723 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:01.348723 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:01.348723 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:01.348723 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:01.348723 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:01.348723 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:01.349539 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:01.349539 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:01.349539 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:01.349539 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:01.349539 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:25:01.349539 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:25:01.349539 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:25:01.349539 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:25:01.349539 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:25:01.349539 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:25:01.349539 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:25:01.349539 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:01.352391 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 391/783] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 18 16:25:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 447 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:01.354248 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d31aaf5-f723-4795-88e3-a3b1d31e4d20 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:01.355560 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d31aaf5-f723-4795-88e3-a3b1d31e4d20 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:01.356086 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d31aaf5-f723-4795-88e3-a3b1d31e4d20 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:01.357494 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 392/783] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Jul 18 16:25:01 2022] GET /identity/v3/services => generated 772 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:01.358580 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afbb5a2-6b90-4f78-98ea-31d3638d4890 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:01.358816 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 392/784] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:25:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:01.369058 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547a719-0ee0-4c48-804c-8af900b19204 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:01.369372 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547a719-0ee0-4c48-804c-8af900b19204 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:01.370129 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547a719-0ee0-4c48-804c-8af900b19204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:01.403196 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-7547a719-0ee0-4c48-804c-8af900b19204 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:01.404568 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-7547a719-0ee0-4c48-804c-8af900b19204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:01.450304 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:01.571977 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 393/785] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:01.574674 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afbb5a2-6b90-4f78-98ea-31d3638d4890 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:01.609342 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7547a719-0ee0-4c48-804c-8af900b19204 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:01.622046 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3d10d6-d9a3-4da8-b0aa-1dcba395f94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:01.622845 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3d10d6-d9a3-4da8-b0aa-1dcba395f94b None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:01.623130 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3d10d6-d9a3-4da8-b0aa-1dcba395f94b None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:01.623405 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3d10d6-d9a3-4da8-b0aa-1dcba395f94b None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:01.625853 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3d10d6-d9a3-4da8-b0aa-1dcba395f94b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:01.626007 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3d10d6-d9a3-4da8-b0aa-1dcba395f94b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:01.627904 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3d10d6-d9a3-4da8-b0aa-1dcba395f94b None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:01.637938 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 393/786] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:25:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:01.643167 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3d10d6-d9a3-4da8-b0aa-1dcba395f94b None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:01.752892 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7547a719-0ee0-4c48-804c-8af900b19204 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:01.771175 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:01.796563 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f434e104-cf6b-4ab8-b98b-236024de3f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:01.797387 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f434e104-cf6b-4ab8-b98b-236024de3f08 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:01.797675 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f434e104-cf6b-4ab8-b98b-236024de3f08 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:01.798109 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f434e104-cf6b-4ab8-b98b-236024de3f08 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:01.811577 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f434e104-cf6b-4ab8-b98b-236024de3f08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:25:01.825476 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-f434e104-cf6b-4ab8-b98b-236024de3f08 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:25:01.825476 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:01.825476 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:01.825476 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:01.825476 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:01.825476 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:01.825476 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:01.825476 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:01.825476 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:01.825476 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:01.825476 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:01.825476 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:01.825476 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:25:01.825476 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:25:01.825476 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:25:01.825476 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:25:01.826577 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:25:01.826577 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:25:01.826577 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:25:01.826577 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:25:01.826577 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:01.826577 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:01.826577 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:01.826577 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:01.826577 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:01.826577 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:01.826577 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:01.826577 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:01.826577 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:01.826577 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:01.826577 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:01.826577 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:01.827585 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:01.827585 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:01.827585 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:01.827585 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:01.827585 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:25:01.827585 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:25:01.827585 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:25:01.827585 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:25:01.827585 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:25:01.827585 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:25:01.827585 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:25:01.827585 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:01.831015 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 394/787] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 18 16:25:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:25:01.842378 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3d10d6-d9a3-4da8-b0aa-1dcba395f94b None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:01.843887 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f434e104-cf6b-4ab8-b98b-236024de3f08 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:01.847737 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:01.957795 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a66245b-ba78-4e3d-96ca-f3f6d0ce682a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:01.959054 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a66245b-ba78-4e3d-96ca-f3f6d0ce682a None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:01.959580 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a66245b-ba78-4e3d-96ca-f3f6d0ce682a None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:01.960109 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a66245b-ba78-4e3d-96ca-f3f6d0ce682a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:01.966379 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f434e104-cf6b-4ab8-b98b-236024de3f08 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:01.996488 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a66245b-ba78-4e3d-96ca-f3f6d0ce682a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:01.998365 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a66245b-ba78-4e3d-96ca-f3f6d0ce682a None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:02.006058 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c54127e-c0ae-4386-bc46-635631e76410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:02.006743 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54127e-c0ae-4386-bc46-635631e76410 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:02.007031 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54127e-c0ae-4386-bc46-635631e76410 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:02.007311 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54127e-c0ae-4386-bc46-635631e76410 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:02.008466 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c54127e-c0ae-4386-bc46-635631e76410 None admin] RBAC: Adding query filter params (name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:02.008787 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c54127e-c0ae-4386-bc46-635631e76410 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:02.010217 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c54127e-c0ae-4386-bc46-635631e76410 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:02.020631 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 395/788] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 18 16:25:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:02.027503 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c54127e-c0ae-4386-bc46-635631e76410 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:02.037597 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-5a66245b-ba78-4e3d-96ca-f3f6d0ce682a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:02.037597 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:02.037597 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:02.037597 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:02.037597 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:02.037597 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:02.037597 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:02.037597 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:02.037597 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:02.037597 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:02.037597 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:02.037597 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:02.037597 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:25:02.037597 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:25:02.037597 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:25:02.038766 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:25:02.038766 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:02.038766 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:02.038766 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:02.038766 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:02.038766 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:02.038766 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:02.038766 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:02.038766 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:02.038766 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:02.038766 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:02.038766 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:02.038766 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:02.038766 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:02.038766 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:02.038766 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:02.038766 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:02.040041 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:25:02.040041 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:25:02.040041 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:02.040041 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:02.047676 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 394/789] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 400 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:02.052406 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a66245b-ba78-4e3d-96ca-f3f6d0ce682a None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:02.088005 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:02.182212 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a66245b-ba78-4e3d-96ca-f3f6d0ce682a None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:02.203401 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c54127e-c0ae-4386-bc46-635631e76410 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:02.241193 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bbdbc4-f999-4a4e-996c-cf2efc274fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:02.241979 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b342a1d-f700-4e9c-b963-28197001a667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:02.242098 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bbdbc4-f999-4a4e-996c-cf2efc274fff None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:02.242373 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bbdbc4-f999-4a4e-996c-cf2efc274fff None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:02.242656 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bbdbc4-f999-4a4e-996c-cf2efc274fff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:02.242768 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b342a1d-f700-4e9c-b963-28197001a667 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:02.242921 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b342a1d-f700-4e9c-b963-28197001a667 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:02.243198 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b342a1d-f700-4e9c-b963-28197001a667 None admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:02.243954 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b342a1d-f700-4e9c-b963-28197001a667 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:02.244256 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b342a1d-f700-4e9c-b963-28197001a667 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:02.246466 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b342a1d-f700-4e9c-b963-28197001a667 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:02.264883 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 395/790] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:25:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:02.266483 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b342a1d-f700-4e9c-b963-28197001a667 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:02.268307 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bbdbc4-f999-4a4e-996c-cf2efc274fff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:02.269821 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bbdbc4-f999-4a4e-996c-cf2efc274fff None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:02.284299 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:02.302156 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-58bbdbc4-f999-4a4e-996c-cf2efc274fff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:02.302156 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:02.302156 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:02.302156 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:02.302156 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:02.302156 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:02.302156 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:02.302156 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:02.302156 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:02.302156 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:02.302156 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:02.302156 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:02.302156 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:25:02.302156 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:25:02.302156 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:25:02.303154 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:25:02.303154 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:02.303154 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:02.303154 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:02.303154 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:02.303154 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:02.303154 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:02.303154 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:02.303154 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:02.303154 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:02.303154 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:02.303154 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:02.303154 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:02.303154 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:02.303154 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:02.303154 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:02.303154 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:02.304103 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:25:02.304103 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:25:02.304103 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:02.304103 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:02.307672 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 396/791] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:25:02.317362 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bbdbc4-f999-4a4e-996c-cf2efc274fff None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:02.411000 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b342a1d-f700-4e9c-b963-28197001a667 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:02.467077 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96423b9f-1640-4ab5-9f84-e7d231096e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:02.467933 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96423b9f-1640-4ab5-9f84-e7d231096e23 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:02.468214 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96423b9f-1640-4ab5-9f84-e7d231096e23 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:02.468487 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96423b9f-1640-4ab5-9f84-e7d231096e23 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:02.480031 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96423b9f-1640-4ab5-9f84-e7d231096e23 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:02.481569 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96423b9f-1640-4ab5-9f84-e7d231096e23 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:02.516552 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-96423b9f-1640-4ab5-9f84-e7d231096e23 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 16:25:02.516552 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:02.516552 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:02.516552 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:02.516552 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:02.516552 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:02.516552 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:02.516552 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:02.516552 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:02.516552 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:02.516552 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:02.516552 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:02.516552 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:25:02.516552 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:25:02.516552 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:25:02.516552 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:25:02.517576 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:02.517576 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:02.517576 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:25:02.517576 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:02.517576 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:25:02.517576 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:02.517576 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:02.517576 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:02.517576 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:02.517576 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:02.517576 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:02.517576 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:02.517576 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:02.517576 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:02.517576 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:02.517576 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:02.517576 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:02.517576 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:02.518503 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:02.518503 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:02.518503 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:25:02.518503 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:25:02.518503 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:25:02.518503 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:25:02.518503 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:25:02.518503 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:25:02.518503 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 16:25:02.518503 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:02.518503 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 396/792] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 18 16:25:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:02.525317 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96423b9f-1640-4ab5-9f84-e7d231096e23 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:02.638054 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:02.666176 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96423b9f-1640-4ab5-9f84-e7d231096e23 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:02.674717 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:02.688073 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bbdbc4-f999-4a4e-996c-cf2efc274fff None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:02.706521 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cab433c-feb7-4f9d-9a9a-f9b0c73f23a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:02.706521 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab433c-feb7-4f9d-9a9a-f9b0c73f23a7 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:02.706521 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab433c-feb7-4f9d-9a9a-f9b0c73f23a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:02.706521 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab433c-feb7-4f9d-9a9a-f9b0c73f23a7 None admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:02.707224 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cab433c-feb7-4f9d-9a9a-f9b0c73f23a7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:02.707224 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cab433c-feb7-4f9d-9a9a-f9b0c73f23a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:02.707823 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cab433c-feb7-4f9d-9a9a-f9b0c73f23a7 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:02.727621 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2e4ae9-9ab6-4432-904d-fa99dace93d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:02.728299 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e4ae9-9ab6-4432-904d-fa99dace93d6 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:02.730061 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e4ae9-9ab6-4432-904d-fa99dace93d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:02.730061 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e4ae9-9ab6-4432-904d-fa99dace93d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:02.730375 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2e4ae9-9ab6-4432-904d-fa99dace93d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:02.730575 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2e4ae9-9ab6-4432-904d-fa99dace93d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:02.732276 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2e4ae9-9ab6-4432-904d-fa99dace93d6 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:02.733789 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 397/793] 10.222.0.26 () {56 vars in 1135 bytes} [Mon Jul 18 16:25:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:02.735297 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba0249a-fb6a-4387-bfc1-3fc41b5cc911 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:02.735631 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba0249a-fb6a-4387-bfc1-3fc41b5cc911 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:02.735975 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba0249a-fb6a-4387-bfc1-3fc41b5cc911 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:02.738056 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 398/794] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:25:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:02.738898 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba0249a-fb6a-4387-bfc1-3fc41b5cc911 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:02.746628 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 397/795] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:25:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 430 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:02.753132 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5894d-a0f7-4365-a807-988a3d339d8b None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:02.753623 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5894d-a0f7-4365-a807-988a3d339d8b None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:02.754282 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5894d-a0f7-4365-a807-988a3d339d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:02.772822 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-ceb5894d-a0f7-4365-a807-988a3d339d8b None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:02.774312 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-ceb5894d-a0f7-4365-a807-988a3d339d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:02.858252 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba0249a-fb6a-4387-bfc1-3fc41b5cc911 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:02.865251 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 398/796] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:02.866235 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb5894d-a0f7-4365-a807-988a3d339d8b None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:02.892374 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:02.897976 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ec016-02e4-47cb-8f0e-2a52b6c4a463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:02.899092 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ec016-02e4-47cb-8f0e-2a52b6c4a463 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:02.899598 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ec016-02e4-47cb-8f0e-2a52b6c4a463 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:02.901518 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ec016-02e4-47cb-8f0e-2a52b6c4a463 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:02.911780 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121ec016-02e4-47cb-8f0e-2a52b6c4a463 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:02.913748 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121ec016-02e4-47cb-8f0e-2a52b6c4a463 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:02.933663 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-121ec016-02e4-47cb-8f0e-2a52b6c4a463 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:02.933663 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:02.933663 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:02.933663 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:02.933663 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:02.933663 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:02.933663 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:02.933663 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:02.933663 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:02.933663 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:02.933663 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:02.933663 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:02.933663 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:25:02.933663 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:25:02.933663 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:25:02.934571 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:25:02.934571 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:02.934571 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:02.934571 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:02.934571 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:02.934571 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:02.934571 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:02.934571 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:02.934571 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:02.934571 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:02.934571 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:02.934571 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:02.934571 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:02.934571 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:02.934571 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:02.934571 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:02.934571 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:02.935458 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:25:02.935458 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:25:02.935458 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:02.935458 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:02.942166 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 399/797] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:02.943321 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ec016-02e4-47cb-8f0e-2a52b6c4a463 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:02.958947 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:02.983872 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb5894d-a0f7-4365-a807-988a3d339d8b None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:03.022420 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e7eb22-1c6f-4ad6-a214-df965c27ca07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:03.023231 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e7eb22-1c6f-4ad6-a214-df965c27ca07 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:03.023539 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e7eb22-1c6f-4ad6-a214-df965c27ca07 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:03.023808 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e7eb22-1c6f-4ad6-a214-df965c27ca07 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:03.032620 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e7eb22-1c6f-4ad6-a214-df965c27ca07 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:03.034474 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e7eb22-1c6f-4ad6-a214-df965c27ca07 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:03.050331 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-62e7eb22-1c6f-4ad6-a214-df965c27ca07 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 16:25:03.050331 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:03.050331 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:03.050331 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:03.050331 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:03.050331 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:03.050331 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:03.050331 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:03.050331 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:03.050331 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:03.050331 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:03.050331 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:03.050331 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:25:03.050331 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:25:03.050331 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:25:03.050331 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:25:03.051558 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:03.051558 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:03.051558 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:25:03.051558 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:03.051558 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:25:03.051558 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:03.051558 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:03.051558 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:03.051558 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:03.051558 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:03.051558 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:03.051558 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:03.051558 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:03.051558 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:03.051558 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:03.051558 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:03.051558 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:03.051558 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:03.052514 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:03.052514 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:03.052514 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:25:03.052514 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:25:03.052514 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:25:03.052514 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:25:03.052514 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:25:03.052514 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:25:03.052514 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 16:25:03.052514 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:03.052514 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 399/798] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Jul 18 16:25:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:03.052514 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e7eb22-1c6f-4ad6-a214-df965c27ca07 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:03.059529 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ec016-02e4-47cb-8f0e-2a52b6c4a463 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:03.102414 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a68fcd-7841-4e31-b4d6-4bea71fbdd06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:03.103527 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a68fcd-7841-4e31-b4d6-4bea71fbdd06 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:03.104469 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a68fcd-7841-4e31-b4d6-4bea71fbdd06 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:03.104755 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a68fcd-7841-4e31-b4d6-4bea71fbdd06 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:03.122199 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a68fcd-7841-4e31-b4d6-4bea71fbdd06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:25:03.132299 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-47a68fcd-7841-4e31-b4d6-4bea71fbdd06 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:25:03.132299 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:03.132299 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:03.132299 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:03.132299 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:03.132299 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:03.132299 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:03.132299 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:03.132299 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:03.132299 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:03.132299 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:03.132299 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:03.132299 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:25:03.132299 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:25:03.132299 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:25:03.132299 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:25:03.133346 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:25:03.133346 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:25:03.133346 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:25:03.133346 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:25:03.133346 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:03.133346 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:03.133346 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:03.133346 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:03.133346 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:03.133346 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:03.133346 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:03.133346 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:03.133346 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:03.133346 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:03.133346 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:03.133346 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:03.134269 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:03.134269 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:03.134269 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:03.134269 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:03.134269 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:25:03.134269 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:25:03.134269 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:25:03.134269 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:25:03.134269 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:25:03.134269 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:25:03.134269 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:25:03.134269 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:03.134903 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 400/799] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 18 16:25:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:03.139556 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b7c978-800b-4fa1-b39b-fe0c026477ee None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:03.139556 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b7c978-800b-4fa1-b39b-fe0c026477ee None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:03.139556 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b7c978-800b-4fa1-b39b-fe0c026477ee None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:03.141973 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 401/800] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:25:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:03.142684 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b7c978-800b-4fa1-b39b-fe0c026477ee None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:03.168049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e7eb22-1c6f-4ad6-a214-df965c27ca07 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:03.197161 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:03.214407 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba36dac-5eea-4751-9741-2c933436ad2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:03.215073 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba36dac-5eea-4751-9741-2c933436ad2b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:03.215361 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba36dac-5eea-4751-9741-2c933436ad2b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:03.215665 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba36dac-5eea-4751-9741-2c933436ad2b None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:03.216448 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba36dac-5eea-4751-9741-2c933436ad2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:03.216751 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba36dac-5eea-4751-9741-2c933436ad2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:03.218112 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba36dac-5eea-4751-9741-2c933436ad2b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:03.225232 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:03.236201 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 400/801] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:25:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:03.237090 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba36dac-5eea-4751-9741-2c933436ad2b None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:03.258260 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b7c978-800b-4fa1-b39b-fe0c026477ee None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:03.296410 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69084a5-18b9-49af-8dad-c165605cf917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:03.297093 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69084a5-18b9-49af-8dad-c165605cf917 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:03.297373 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69084a5-18b9-49af-8dad-c165605cf917 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:03.297647 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69084a5-18b9-49af-8dad-c165605cf917 None admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:03.298483 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69084a5-18b9-49af-8dad-c165605cf917 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:03.298792 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69084a5-18b9-49af-8dad-c165605cf917 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:03.300264 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69084a5-18b9-49af-8dad-c165605cf917 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:03.323870 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 402/802] 10.222.0.26 () {56 vars in 1149 bytes} [Mon Jul 18 16:25:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:03.323870 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39151320-40a7-4d1d-bd80-0111da784e3c None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:03.323870 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39151320-40a7-4d1d-bd80-0111da784e3c None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:03.323870 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39151320-40a7-4d1d-bd80-0111da784e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:03.338722 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-39151320-40a7-4d1d-bd80-0111da784e3c None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:03.339966 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-39151320-40a7-4d1d-bd80-0111da784e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:03.350743 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba36dac-5eea-4751-9741-2c933436ad2b None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:03.389971 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129944ad-f30b-4e99-a586-9e70206c3953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:03.390652 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129944ad-f30b-4e99-a586-9e70206c3953 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:03.390934 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129944ad-f30b-4e99-a586-9e70206c3953 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:03.391209 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129944ad-f30b-4e99-a586-9e70206c3953 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:03.392115 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129944ad-f30b-4e99-a586-9e70206c3953 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:03.392418 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129944ad-f30b-4e99-a586-9e70206c3953 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:03.393903 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129944ad-f30b-4e99-a586-9e70206c3953 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:03.408661 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 401/803] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:25:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:03.409347 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129944ad-f30b-4e99-a586-9e70206c3953 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:03.420014 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 403/804] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:03.420794 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39151320-40a7-4d1d-bd80-0111da784e3c None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:03.465497 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:03.484847 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:03.530510 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129944ad-f30b-4e99-a586-9e70206c3953 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:03.547095 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39151320-40a7-4d1d-bd80-0111da784e3c None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:03.566898 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3f3882-3553-4bb8-a112-78dfa98b867e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:03.567725 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3f3882-3553-4bb8-a112-78dfa98b867e None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:03.568010 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3f3882-3553-4bb8-a112-78dfa98b867e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:03.568282 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3f3882-3553-4bb8-a112-78dfa98b867e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:03.575403 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3f3882-3553-4bb8-a112-78dfa98b867e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:03.576929 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3f3882-3553-4bb8-a112-78dfa98b867e None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:03.584151 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff143e76-71b5-4df3-b0d5-3b6aa21db40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:03.584950 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff143e76-71b5-4df3-b0d5-3b6aa21db40c None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:03.585233 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff143e76-71b5-4df3-b0d5-3b6aa21db40c None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:03.585639 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff143e76-71b5-4df3-b0d5-3b6aa21db40c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:03.591908 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-4a3f3882-3553-4bb8-a112-78dfa98b867e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:25:03.591908 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:03.591908 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:03.591908 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:03.591908 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:03.591908 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:03.591908 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:03.591908 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:03.591908 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:03.591908 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:03.591908 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:03.591908 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:03.591908 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:25:03.591908 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:25:03.591908 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:25:03.592848 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:25:03.592848 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:03.592848 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:03.592848 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:03.592848 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:03.592848 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:03.592848 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:03.592848 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:03.592848 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:03.592848 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:03.592848 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:03.592848 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:03.592848 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:03.592848 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:03.592848 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:03.592848 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:03.592848 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:03.594352 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:25:03.594352 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:25:03.594352 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:25:03.594352 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:25:03.594352 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:25:03.594352 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:25:03.594352 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:25:03.594352 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:03.594352 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 402/805] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:25:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:03.594352 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3f3882-3553-4bb8-a112-78dfa98b867e None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:03.602118 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff143e76-71b5-4df3-b0d5-3b6aa21db40c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:03.603643 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff143e76-71b5-4df3-b0d5-3b6aa21db40c None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:03.624996 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-ff143e76-71b5-4df3-b0d5-3b6aa21db40c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:03.624996 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:03.624996 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:03.624996 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:03.624996 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:03.624996 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:03.624996 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:03.624996 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:03.624996 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:03.624996 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:03.624996 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:03.624996 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:03.624996 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:25:03.624996 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:25:03.624996 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:25:03.625975 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:25:03.625975 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:03.625975 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:03.625975 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:03.625975 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:03.625975 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:03.625975 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:03.625975 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:03.625975 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:03.625975 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:03.625975 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:03.625975 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:03.625975 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:03.625975 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:03.625975 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:03.625975 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:03.625975 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:03.626869 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:25:03.626869 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:25:03.626869 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:03.626869 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:03.626869 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 404/806] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:03.627151 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff143e76-71b5-4df3-b0d5-3b6aa21db40c None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:03.708605 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3f3882-3553-4bb8-a112-78dfa98b867e None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:03.717240 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:03.731429 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:03.733615 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff143e76-71b5-4df3-b0d5-3b6aa21db40c None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:03.749086 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c2f284-2a7b-4500-98bc-ef8e522860ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:03.749883 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c2f284-2a7b-4500-98bc-ef8e522860ed None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:03.750165 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c2f284-2a7b-4500-98bc-ef8e522860ed None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:03.750440 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c2f284-2a7b-4500-98bc-ef8e522860ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:03.762104 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c2f284-2a7b-4500-98bc-ef8e522860ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:03.763568 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c2f284-2a7b-4500-98bc-ef8e522860ed None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:03.770310 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6453a2ba-cd43-458b-b2a5-bf78439a25d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:03.771123 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6453a2ba-cd43-458b-b2a5-bf78439a25d8 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:03.771405 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6453a2ba-cd43-458b-b2a5-bf78439a25d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:03.771711 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6453a2ba-cd43-458b-b2a5-bf78439a25d8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:03.772366 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6453a2ba-cd43-458b-b2a5-bf78439a25d8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:03.773726 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6453a2ba-cd43-458b-b2a5-bf78439a25d8 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:03.782456 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-e7c2f284-2a7b-4500-98bc-ef8e522860ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:03.782456 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:03.782456 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:03.782456 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:03.782456 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:03.782456 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:03.782456 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:03.782456 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:03.782456 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:03.782456 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:03.782456 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:03.782456 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:03.782456 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:25:03.782456 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:25:03.782456 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:25:03.783547 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:25:03.783547 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:03.783547 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:03.783547 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:03.783547 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:03.783547 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:03.783547 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:03.783547 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:03.783547 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:03.783547 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:03.783547 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:03.783547 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:03.783547 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:03.783547 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:03.783547 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:03.783547 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:03.783547 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:03.784663 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:25:03.784663 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:25:03.784663 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:03.784663 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:03.786705 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-6453a2ba-cd43-458b-b2a5-bf78439a25d8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 16:25:03.786705 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:03.786705 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:25:03.786705 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:25:03.786705 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:25:03.786705 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:25:03.786705 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:25:03.786705 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:03.786705 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 16:25:03.786705 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:03.786705 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:25:03.786705 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:03.786705 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:03.786705 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:03.786705 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:03.786705 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:03.787938 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:03.787938 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:03.787938 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:03.787938 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:03.787938 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:03.787938 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:03.787938 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:03.787938 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:25:03.787938 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:25:03.787938 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:25:03.787938 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:25:03.787938 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:03.787938 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:03.787938 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:03.787938 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:03.787938 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:03.789093 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:03.789093 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:03.789093 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:03.789093 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:03.789093 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:03.789093 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:03.789093 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:03.789093 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:03.789093 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:03.789093 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:03.789093 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:03.789093 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:25:03.789093 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:03.789093 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 16:25:03.789093 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:03.789093 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 405/807] 10.222.0.26 () {56 vars in 1129 bytes} [Mon Jul 18 16:25:03 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:03.790284 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6453a2ba-cd43-458b-b2a5-bf78439a25d8 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:03.796043 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 403/808] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:03.796829 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c2f284-2a7b-4500-98bc-ef8e522860ed None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:03.905183 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6453a2ba-cd43-458b-b2a5-bf78439a25d8 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:03.905571 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c2f284-2a7b-4500-98bc-ef8e522860ed None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:03.941103 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ea356-b6ca-4fbd-a34b-aee5662736f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:03.941324 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d3ddcd-7b47-44be-8b9f-31a9b26580a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:03.941769 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ea356-b6ca-4fbd-a34b-aee5662736f3 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:03.942146 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d3ddcd-7b47-44be-8b9f-31a9b26580a3 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:03.942260 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ea356-b6ca-4fbd-a34b-aee5662736f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:03.942590 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ea356-b6ca-4fbd-a34b-aee5662736f3 None admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:03.942691 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d3ddcd-7b47-44be-8b9f-31a9b26580a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:03.943045 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d3ddcd-7b47-44be-8b9f-31a9b26580a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:03.943384 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761ea356-b6ca-4fbd-a34b-aee5662736f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:03.943723 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761ea356-b6ca-4fbd-a34b-aee5662736f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:03.944055 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d3ddcd-7b47-44be-8b9f-31a9b26580a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:03.944453 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d3ddcd-7b47-44be-8b9f-31a9b26580a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:03.945151 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761ea356-b6ca-4fbd-a34b-aee5662736f3 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:03.945987 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d3ddcd-7b47-44be-8b9f-31a9b26580a3 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:03.967662 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 404/809] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:25:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:03.967863 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 406/810] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 18 16:25:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:03.967960 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d3ddcd-7b47-44be-8b9f-31a9b26580a3 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:03.968052 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ea356-b6ca-4fbd-a34b-aee5662736f3 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:03.977783 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:04.006847 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:04.078225 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ea356-b6ca-4fbd-a34b-aee5662736f3 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:04.079598 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d3ddcd-7b47-44be-8b9f-31a9b26580a3 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:04.112540 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e52a439-fa87-498e-8dc0-e78ef6972f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:04.113209 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52a439-fa87-498e-8dc0-e78ef6972f18 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:04.113417 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989aa0e1-0b4a-4745-9397-9ccf6a03084d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:04.113545 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52a439-fa87-498e-8dc0-e78ef6972f18 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:04.113788 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52a439-fa87-498e-8dc0-e78ef6972f18 None admin] PATH_INFO: `/v3/services` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:04.114243 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989aa0e1-0b4a-4745-9397-9ccf6a03084d None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:04.114493 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e52a439-fa87-498e-8dc0-e78ef6972f18 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:04.114587 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989aa0e1-0b4a-4745-9397-9ccf6a03084d None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:04.114798 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e52a439-fa87-498e-8dc0-e78ef6972f18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:04.114898 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989aa0e1-0b4a-4745-9397-9ccf6a03084d None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:04.115537 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989aa0e1-0b4a-4745-9397-9ccf6a03084d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:04.115855 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989aa0e1-0b4a-4745-9397-9ccf6a03084d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:04.116198 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e52a439-fa87-498e-8dc0-e78ef6972f18 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:04.117228 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989aa0e1-0b4a-4745-9397-9ccf6a03084d None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:04.121491 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 407/811] 10.222.0.26 () {56 vars in 1124 bytes} [Mon Jul 18 16:25:03 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:04.123083 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e52a439-fa87-498e-8dc0-e78ef6972f18 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:04.130584 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 405/812] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:25:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:04.131415 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989aa0e1-0b4a-4745-9397-9ccf6a03084d None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:04.196737 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:04.223157 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:04.227813 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e52a439-fa87-498e-8dc0-e78ef6972f18 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:04.232188 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989aa0e1-0b4a-4745-9397-9ccf6a03084d None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:04.262809 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529604c7-3b22-48e3-b6b7-a9785e5ec378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:04.263636 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529604c7-3b22-48e3-b6b7-a9785e5ec378 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:04.263929 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529604c7-3b22-48e3-b6b7-a9785e5ec378 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:04.264206 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529604c7-3b22-48e3-b6b7-a9785e5ec378 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:04.265855 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c30e0-377f-4e04-bc4a-a8476d033648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:04.266733 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c30e0-377f-4e04-bc4a-a8476d033648 None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:04.267019 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c30e0-377f-4e04-bc4a-a8476d033648 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:04.267291 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c30e0-377f-4e04-bc4a-a8476d033648 None admin] PATH_INFO: `/v3/projects/eb9747e2299c4e0991fd7edb4462c7e5/users/43854d7a6df2466eb7976fabe6011767/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:04.269314 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73c30e0-377f-4e04-bc4a-a8476d033648 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb9747e2299c4e0991fd7edb4462c7e5, user_id=43854d7a6df2466eb7976fabe6011767, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:04.270746 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73c30e0-377f-4e04-bc4a-a8476d033648 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:04.271133 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529604c7-3b22-48e3-b6b7-a9785e5ec378 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:04.272601 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529604c7-3b22-48e3-b6b7-a9785e5ec378 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:04.287472 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-529604c7-3b22-48e3-b6b7-a9785e5ec378 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:25:04.287472 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:04.287472 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:04.287472 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:04.287472 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:04.287472 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:04.287472 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:04.287472 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:04.287472 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:04.287472 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:04.287472 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:04.287472 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:04.287472 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:25:04.287472 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:25:04.287472 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:25:04.289160 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:25:04.289160 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:04.289160 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:04.289160 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:04.289160 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:04.289160 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:04.289160 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:04.289160 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:04.289160 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:04.289160 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:04.289160 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:04.289160 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:04.289160 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:04.289160 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:04.289160 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:04.289160 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:04.289160 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:04.291893 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:25:04.291893 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:25:04.291893 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:25:04.291893 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:25:04.291893 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:25:04.291893 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:25:04.291893 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:25:04.291893 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:04.291893 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 408/813] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:25:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:04.291893 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c9662c-cf51-47e2-95e6-85aa919651f2 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:04.291893 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c9662c-cf51-47e2-95e6-85aa919651f2 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:04.293492 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c9662c-cf51-47e2-95e6-85aa919651f2 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:04.293492 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 409/814] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:25:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:04.295370 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c9662c-cf51-47e2-95e6-85aa919651f2 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:04.350062 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 406/815] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 18 16:25:04 2022] PUT /identity/v3/projects/eb9747e2299c4e0991fd7edb4462c7e5/users/43854d7a6df2466eb7976fabe6011767/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:25:04.350062 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c30e0-377f-4e04-bc4a-a8476d033648 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:04.407826 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c9662c-cf51-47e2-95e6-85aa919651f2 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:04.442511 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:04.447979 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7405a57d-52cb-402c-a67e-23dd2af1b7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:04.448689 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7405a57d-52cb-402c-a67e-23dd2af1b7ce None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:04.448969 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7405a57d-52cb-402c-a67e-23dd2af1b7ce None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:04.449242 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7405a57d-52cb-402c-a67e-23dd2af1b7ce None admin] PATH_INFO: `/v3/services` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:04.449962 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7405a57d-52cb-402c-a67e-23dd2af1b7ce None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:04.450279 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7405a57d-52cb-402c-a67e-23dd2af1b7ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:04.451697 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7405a57d-52cb-402c-a67e-23dd2af1b7ce None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:04.461978 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 410/816] 10.222.0.26 () {56 vars in 1124 bytes} [Mon Jul 18 16:25:04 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:04.461978 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a6d07-1a68-4db8-8f9c-6f7bff91c370 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:04.461978 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a6d07-1a68-4db8-8f9c-6f7bff91c370 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:04.461978 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a6d07-1a68-4db8-8f9c-6f7bff91c370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:04.479381 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-9a4a6d07-1a68-4db8-8f9c-6f7bff91c370 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:04.480653 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-9a4a6d07-1a68-4db8-8f9c-6f7bff91c370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:04.493334 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c30e0-377f-4e04-bc4a-a8476d033648 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:04.541955 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74bd058-4014-4cc0-a3f2-c098980faf46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:04.543413 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74bd058-4014-4cc0-a3f2-c098980faf46 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:04.543723 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74bd058-4014-4cc0-a3f2-c098980faf46 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:04.543993 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74bd058-4014-4cc0-a3f2-c098980faf46 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:04.552027 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74bd058-4014-4cc0-a3f2-c098980faf46 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:04.553531 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74bd058-4014-4cc0-a3f2-c098980faf46 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:04.569190 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-f74bd058-4014-4cc0-a3f2-c098980faf46 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:25:04.569190 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:04.569190 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:04.569190 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:04.569190 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:04.569190 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:04.569190 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:04.569190 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:04.569190 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:04.569190 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:04.569190 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:04.569190 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:04.569190 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:25:04.569190 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:25:04.569190 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:25:04.569190 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:25:04.570184 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:04.570184 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:04.570184 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:25:04.570184 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:04.570184 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:25:04.570184 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:04.570184 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:04.570184 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:04.570184 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:04.570184 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:04.570184 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:04.570184 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:04.570184 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:04.570184 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:04.570184 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:04.570184 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:04.570184 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:04.570184 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:04.571226 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:04.571226 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:04.571226 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:25:04.571226 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:25:04.571226 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:25:04.571226 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:25:04.571226 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:25:04.571226 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:25:04.571226 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:25:04.571226 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:04.571226 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 407/817] 10.222.0.26 () {56 vars in 1093 bytes} [Mon Jul 18 16:25:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:04.571857 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74bd058-4014-4cc0-a3f2-c098980faf46 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:04.591517 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 411/818] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:04.591517 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a6d07-1a68-4db8-8f9c-6f7bff91c370 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:04.594450 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:04.689860 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74bd058-4014-4cc0-a3f2-c098980faf46 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:04.698265 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a6d07-1a68-4db8-8f9c-6f7bff91c370 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:04.725266 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:04.734312 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85cef98-18ed-4e0d-b9f5-d0a7bb82c45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:04.734747 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38009e84-6de2-4f6d-bfd2-eb5965ee1a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:04.734903 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85cef98-18ed-4e0d-b9f5-d0a7bb82c45f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:04.735182 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85cef98-18ed-4e0d-b9f5-d0a7bb82c45f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:04.735412 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38009e84-6de2-4f6d-bfd2-eb5965ee1a58 None admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:04.735522 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85cef98-18ed-4e0d-b9f5-d0a7bb82c45f None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:04.735763 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38009e84-6de2-4f6d-bfd2-eb5965ee1a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:04.736044 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38009e84-6de2-4f6d-bfd2-eb5965ee1a58 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:04.736273 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85cef98-18ed-4e0d-b9f5-d0a7bb82c45f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:04.736580 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85cef98-18ed-4e0d-b9f5-d0a7bb82c45f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:04.736787 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38009e84-6de2-4f6d-bfd2-eb5965ee1a58 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:04.738004 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85cef98-18ed-4e0d-b9f5-d0a7bb82c45f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:04.738180 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38009e84-6de2-4f6d-bfd2-eb5965ee1a58 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:04.751177 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 408/819] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 18 16:25:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:04.752046 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85cef98-18ed-4e0d-b9f5-d0a7bb82c45f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:04.833324 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:04.864957 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85cef98-18ed-4e0d-b9f5-d0a7bb82c45f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:04.871811 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 412/820] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Jul 18 16:25:04 2022] POST /identity/v3/endpoints => generated 346 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:25:04.871811 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38009e84-6de2-4f6d-bfd2-eb5965ee1a58 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:04.921094 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ea5338-11cf-4f3a-8543-6cf6f1bc5be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:04.922875 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea5338-11cf-4f3a-8543-6cf6f1bc5be6 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:04.923183 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea5338-11cf-4f3a-8543-6cf6f1bc5be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:04.923434 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea5338-11cf-4f3a-8543-6cf6f1bc5be6 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:04.924560 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ea5338-11cf-4f3a-8543-6cf6f1bc5be6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:04.924880 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ea5338-11cf-4f3a-8543-6cf6f1bc5be6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:04.926481 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ea5338-11cf-4f3a-8543-6cf6f1bc5be6 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:04.954538 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 409/821] 10.222.0.26 () {56 vars in 1133 bytes} [Mon Jul 18 16:25:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:04.955823 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ea5338-11cf-4f3a-8543-6cf6f1bc5be6 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:05.030851 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38009e84-6de2-4f6d-bfd2-eb5965ee1a58 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:05.075916 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ea5338-11cf-4f3a-8543-6cf6f1bc5be6 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:05.081451 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ff7fe-4df5-418d-ba22-860d2dab08b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:05.082268 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ff7fe-4df5-418d-ba22-860d2dab08b3 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:05.082551 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ff7fe-4df5-418d-ba22-860d2dab08b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:05.082825 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ff7fe-4df5-418d-ba22-860d2dab08b3 None admin] PATH_INFO: `/v3/services/network` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:05.083509 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0ff7fe-4df5-418d-ba22-860d2dab08b3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:05.084905 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0ff7fe-4df5-418d-ba22-860d2dab08b3 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:05.098296 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-ef0ff7fe-4df5-418d-ba22-860d2dab08b3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 16:25:05.098296 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:05.098296 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:25:05.098296 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:25:05.098296 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:25:05.098296 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:25:05.098296 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:25:05.098296 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:05.098296 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 16:25:05.098296 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:05.098296 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:25:05.098296 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:05.098296 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:05.098296 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:05.098296 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:05.098296 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:05.098296 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:05.099581 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:05.099581 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:05.099581 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:05.099581 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:05.099581 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:05.099581 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:05.099581 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:25:05.099581 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:25:05.099581 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:25:05.099581 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:25:05.099581 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:05.099581 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:05.099581 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:05.099581 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:05.099581 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:05.099581 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:05.100483 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:05.100483 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:05.100483 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:05.100483 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:05.100483 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:05.100483 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:05.100483 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:05.100483 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:05.100483 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:05.100483 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:05.100483 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:25:05.100483 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:05.100483 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 16:25:05.100483 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:05.105533 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 413/822] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:25:04 2022] GET /identity/v3/services/network => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:05.106443 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ff7fe-4df5-418d-ba22-860d2dab08b3 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:05.134005 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:05.151453 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a40573-4735-4ea5-8c41-7d31d5becc2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:05.152491 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a40573-4735-4ea5-8c41-7d31d5becc2f None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:05.152776 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a40573-4735-4ea5-8c41-7d31d5becc2f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:05.153052 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a40573-4735-4ea5-8c41-7d31d5becc2f None admin] PATH_INFO: `/v3/projects/eb9747e2299c4e0991fd7edb4462c7e5/users/da799b0044f54da6835aa06e43ce1a6f/roles/4fe970e98f394bbfb096c4879bebdf97` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:05.180641 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a40573-4735-4ea5-8c41-7d31d5becc2f None admin] RBAC: Authorizing `identity:create_grant(project_id=eb9747e2299c4e0991fd7edb4462c7e5, user_id=da799b0044f54da6835aa06e43ce1a6f, role_id=4fe970e98f394bbfb096c4879bebdf97)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:05.183070 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a40573-4735-4ea5-8c41-7d31d5becc2f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:05.194623 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:05.255155 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ff7fe-4df5-418d-ba22-860d2dab08b3 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:05.274389 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 410/823] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 18 16:25:04 2022] PUT /identity/v3/projects/eb9747e2299c4e0991fd7edb4462c7e5/users/da799b0044f54da6835aa06e43ce1a6f/roles/4fe970e98f394bbfb096c4879bebdf97 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:25:05.276458 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a40573-4735-4ea5-8c41-7d31d5becc2f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:05.346583 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e095c65c-0594-4a71-85fe-d55d743a3d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:05.347815 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e095c65c-0594-4a71-85fe-d55d743a3d7a None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:05.348303 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e095c65c-0594-4a71-85fe-d55d743a3d7a None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:05.348798 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e095c65c-0594-4a71-85fe-d55d743a3d7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:05.534075 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e095c65c-0594-4a71-85fe-d55d743a3d7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:05.536946 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e095c65c-0594-4a71-85fe-d55d743a3d7a None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:05.538561 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a40573-4735-4ea5-8c41-7d31d5becc2f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:05.608416 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56303a0-168c-4d1f-b760-45161d74c00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:05.609017 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56303a0-168c-4d1f-b760-45161d74c00f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:05.609300 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56303a0-168c-4d1f-b760-45161d74c00f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:05.609568 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56303a0-168c-4d1f-b760-45161d74c00f None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:05.610271 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56303a0-168c-4d1f-b760-45161d74c00f None admin] RBAC: Adding query filter params (name=network) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:05.610569 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56303a0-168c-4d1f-b760-45161d74c00f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:05.614719 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56303a0-168c-4d1f-b760-45161d74c00f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:05.620443 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 411/824] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Jul 18 16:25:05 2022] GET /identity/v3/services?name=network => generated 124 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:05.621951 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:05.632887 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56303a0-168c-4d1f-b760-45161d74c00f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:05.739895 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:05.742621 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 414/825] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:25:05 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 636 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 16:25:05.748002 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e095c65c-0594-4a71-85fe-d55d743a3d7a None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:05.830608 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56303a0-168c-4d1f-b760-45161d74c00f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:05.914847 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e095c65c-0594-4a71-85fe-d55d743a3d7a None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:05.925615 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a05590-d0a0-48ab-afa6-544ddeec0656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:05.926289 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a05590-d0a0-48ab-afa6-544ddeec0656 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:05.926579 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a05590-d0a0-48ab-afa6-544ddeec0656 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:05.927019 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a05590-d0a0-48ab-afa6-544ddeec0656 None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:05.927566 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a05590-d0a0-48ab-afa6-544ddeec0656 None admin] RBAC: Adding query filter params (type=network) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:05.927870 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a05590-d0a0-48ab-afa6-544ddeec0656 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:05.929228 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a05590-d0a0-48ab-afa6-544ddeec0656 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:05.943248 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 412/826] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Jul 18 16:25:05 2022] GET /identity/v3/services?type=network => generated 350 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:05.949618 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a05590-d0a0-48ab-afa6-544ddeec0656 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:05.956296 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a4e8ff-9561-4d42-9f1c-4450b149547f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:05.957364 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a4e8ff-9561-4d42-9f1c-4450b149547f None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:05.957815 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a4e8ff-9561-4d42-9f1c-4450b149547f None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:05.958237 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a4e8ff-9561-4d42-9f1c-4450b149547f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:05.975409 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a4e8ff-9561-4d42-9f1c-4450b149547f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:05.977299 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a4e8ff-9561-4d42-9f1c-4450b149547f None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:05.989353 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:06.015138 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-11a4e8ff-9561-4d42-9f1c-4450b149547f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:06.015138 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:06.015138 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:06.015138 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:06.015138 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:06.015138 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:06.015138 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:06.015138 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:06.015138 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:06.015138 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:06.015138 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:06.015138 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:06.015138 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:25:06.015138 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:25:06.015138 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:25:06.016240 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:25:06.016240 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:06.016240 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:06.016240 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:06.016240 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:06.016240 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:06.016240 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:06.016240 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:06.016240 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:06.016240 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:06.016240 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:06.016240 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:06.016240 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:06.016240 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:06.016240 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:06.016240 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:06.016240 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:06.017330 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:25:06.017330 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:25:06.017330 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:06.017330 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:06.024710 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 415/827] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:06.031254 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a4e8ff-9561-4d42-9f1c-4450b149547f None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:06.125383 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a05590-d0a0-48ab-afa6-544ddeec0656 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:06.127944 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:06.165821 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9eb794-4c3e-47ec-a96f-317b6bdcb1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:06.166532 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9eb794-4c3e-47ec-a96f-317b6bdcb1bc None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:06.167018 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9eb794-4c3e-47ec-a96f-317b6bdcb1bc None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:06.167378 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9eb794-4c3e-47ec-a96f-317b6bdcb1bc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:06.168247 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9eb794-4c3e-47ec-a96f-317b6bdcb1bc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:06.170304 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9eb794-4c3e-47ec-a96f-317b6bdcb1bc None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:06.211614 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a4e8ff-9561-4d42-9f1c-4450b149547f None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:06.250119 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c02c8b-c9f7-4712-804b-3286e6f6f300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:06.251237 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c02c8b-c9f7-4712-804b-3286e6f6f300 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:06.251756 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c02c8b-c9f7-4712-804b-3286e6f6f300 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:06.252513 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c02c8b-c9f7-4712-804b-3286e6f6f300 None admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:06.260463 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 413/828] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Jul 18 16:25:05 2022] POST /identity/v3/endpoints => generated 327 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:25:06.261296 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c02c8b-c9f7-4712-804b-3286e6f6f300 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:06.262221 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c02c8b-c9f7-4712-804b-3286e6f6f300 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:06.267205 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c02c8b-c9f7-4712-804b-3286e6f6f300 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:06.278386 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 416/829] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:25:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:06.286387 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9eb794-4c3e-47ec-a96f-317b6bdcb1bc None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:06.418653 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9eb794-4c3e-47ec-a96f-317b6bdcb1bc None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:06.474282 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadcfab4-1240-46b1-8ff6-be97973a8120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:06.475569 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadcfab4-1240-46b1-8ff6-be97973a8120 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:06.476084 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadcfab4-1240-46b1-8ff6-be97973a8120 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:06.476567 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadcfab4-1240-46b1-8ff6-be97973a8120 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:06.487468 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadcfab4-1240-46b1-8ff6-be97973a8120 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:06.489331 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadcfab4-1240-46b1-8ff6-be97973a8120 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:06.511574 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-aadcfab4-1240-46b1-8ff6-be97973a8120 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:25:06.511574 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:06.511574 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:06.511574 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:06.511574 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:06.511574 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:06.511574 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:06.511574 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:06.511574 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:06.511574 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:06.511574 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:06.511574 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:06.511574 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:25:06.511574 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:25:06.511574 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:25:06.512534 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:25:06.512534 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:06.512534 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:06.512534 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:06.512534 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:06.512534 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:06.512534 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:06.512534 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:06.512534 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:06.512534 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:06.512534 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:06.512534 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:06.512534 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:06.512534 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:06.512534 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:06.512534 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:06.512534 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:06.513501 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:25:06.513501 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:25:06.513501 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:25:06.513501 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:25:06.513501 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:25:06.513501 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:25:06.513501 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:25:06.513501 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:06.516404 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 414/830] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:25:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:06.521746 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c02c8b-c9f7-4712-804b-3286e6f6f300 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:06.588139 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:06.636355 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c02c8b-c9f7-4712-804b-3286e6f6f300 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:06.670950 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c9ec8-16d5-4cf3-9502-47d8ffcdd352 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:06.672110 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c9ec8-16d5-4cf3-9502-47d8ffcdd352 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:06.672615 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c9ec8-16d5-4cf3-9502-47d8ffcdd352 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:06.673205 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268fb87-5f88-4fcc-8ee9-73cc1540df9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:06.673888 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268fb87-5f88-4fcc-8ee9-73cc1540df9b None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:06.674164 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268fb87-5f88-4fcc-8ee9-73cc1540df9b None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:06.674434 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268fb87-5f88-4fcc-8ee9-73cc1540df9b None admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:06.675230 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268fb87-5f88-4fcc-8ee9-73cc1540df9b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:06.676479 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268fb87-5f88-4fcc-8ee9-73cc1540df9b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:06.676720 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 415/831] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:25:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:06.680724 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268fb87-5f88-4fcc-8ee9-73cc1540df9b None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:06.684071 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb6ab1-c059-4c81-abc4-f6d9d4615977 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:06.684595 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb6ab1-c059-4c81-abc4-f6d9d4615977 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:06.685099 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb6ab1-c059-4c81-abc4-f6d9d4615977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:06.692379 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 417/832] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 18 16:25:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:06.699657 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268fb87-5f88-4fcc-8ee9-73cc1540df9b None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:06.706099 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-74cb6ab1-c059-4c81-abc4-f6d9d4615977 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:06.707303 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-74cb6ab1-c059-4c81-abc4-f6d9d4615977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:06.723885 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:06.802430 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 416/833] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:06 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:25:06.836199 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cb6ab1-c059-4c81-abc4-f6d9d4615977 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:06.842211 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268fb87-5f88-4fcc-8ee9-73cc1540df9b None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:06.861039 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:06.880695 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdb2f8e-1efb-452d-82ed-be87385ddbc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:06.881723 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdb2f8e-1efb-452d-82ed-be87385ddbc3 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:06.882244 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdb2f8e-1efb-452d-82ed-be87385ddbc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:06.882813 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdb2f8e-1efb-452d-82ed-be87385ddbc3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:06.883875 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdb2f8e-1efb-452d-82ed-be87385ddbc3 None admin] RBAC: Adding query filter params (role.id=98eaadb3669f4b608d572c001742f559, scope.project.id=eb9747e2299c4e0991fd7edb4462c7e5, user.id=5b868bfa44b94ca5b470166d39f20e54) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:06.884914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdb2f8e-1efb-452d-82ed-be87385ddbc3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:06.888039 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdb2f8e-1efb-452d-82ed-be87385ddbc3 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:06.895349 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 418/834] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 18 16:25:06 2022] GET /identity/v3/role_assignments?user.id=5b868bfa44b94ca5b470166d39f20e54&scope.project.id=eb9747e2299c4e0991fd7edb4462c7e5&role.id=98eaadb3669f4b608d572c001742f559 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:06.967983 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cb6ab1-c059-4c81-abc4-f6d9d4615977 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:07.008456 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4f4e07-c993-4684-b522-f1ca8449c3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:07.009280 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f4e07-c993-4684-b522-f1ca8449c3aa None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:07.010191 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f4e07-c993-4684-b522-f1ca8449c3aa None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:07.010465 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f4e07-c993-4684-b522-f1ca8449c3aa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:07.022936 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4f4e07-c993-4684-b522-f1ca8449c3aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:25:07.031182 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-ca4f4e07-c993-4684-b522-f1ca8449c3aa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:25:07.031182 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:07.031182 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:07.031182 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:07.031182 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:07.031182 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:07.031182 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:07.031182 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:07.031182 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:07.031182 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:07.031182 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:07.031182 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:07.031182 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:25:07.031182 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:25:07.031182 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:25:07.031182 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:25:07.032231 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:25:07.032231 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:25:07.032231 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:25:07.032231 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:25:07.032231 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:07.032231 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:07.032231 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:07.032231 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:07.032231 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:07.032231 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:07.032231 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:07.032231 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:07.032231 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:07.032231 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:07.032231 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:07.032231 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:07.033089 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:07.033089 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:07.033089 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:07.033089 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:07.033089 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:25:07.033089 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:25:07.033089 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:25:07.033089 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:25:07.033089 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:25:07.033089 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:25:07.033089 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:25:07.033089 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:07.033089 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 417/835] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 18 16:25:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:07.038673 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdb2f8e-1efb-452d-82ed-be87385ddbc3 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:07.150390 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdb2f8e-1efb-452d-82ed-be87385ddbc3 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:07.187522 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4197109b-f3f3-4436-ab42-48a657fe568c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:07.188202 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4197109b-f3f3-4436-ab42-48a657fe568c None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:07.188489 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4197109b-f3f3-4436-ab42-48a657fe568c None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:07.188767 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4197109b-f3f3-4436-ab42-48a657fe568c None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:07.189527 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4197109b-f3f3-4436-ab42-48a657fe568c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:07.189835 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4197109b-f3f3-4436-ab42-48a657fe568c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:07.191238 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4197109b-f3f3-4436-ab42-48a657fe568c None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:07.201234 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 419/836] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:25:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:07.208661 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4f4e07-c993-4684-b522-f1ca8449c3aa None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:07.219762 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:07.327745 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4f4e07-c993-4684-b522-f1ca8449c3aa None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:07.353455 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149e244-0c3f-4d92-ac8d-4a95b401ea56 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:07.354286 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149e244-0c3f-4d92-ac8d-4a95b401ea56 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:07.354286 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149e244-0c3f-4d92-ac8d-4a95b401ea56 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:07.355636 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 420/837] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:25:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:25:07.361238 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4c6c1a-25d7-4124-9e36-8ea423b374e4 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:07.361793 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4c6c1a-25d7-4124-9e36-8ea423b374e4 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:07.362296 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4c6c1a-25d7-4124-9e36-8ea423b374e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:07.367485 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21adba1-3c89-407a-95c5-5e1d70777fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:07.368301 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21adba1-3c89-407a-95c5-5e1d70777fdb None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:07.368583 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21adba1-3c89-407a-95c5-5e1d70777fdb None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:07.369574 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21adba1-3c89-407a-95c5-5e1d70777fdb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:07.381091 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21adba1-3c89-407a-95c5-5e1d70777fdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:07.382638 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21adba1-3c89-407a-95c5-5e1d70777fdb None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:07.383374 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-3f4c6c1a-25d7-4124-9e36-8ea423b374e4 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:07.385119 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-3f4c6c1a-25d7-4124-9e36-8ea423b374e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:07.401130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-f21adba1-3c89-407a-95c5-5e1d70777fdb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:07.401130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:07.401130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:07.401130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:07.401130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:07.401130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:07.401130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:07.401130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:07.401130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:07.401130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:07.401130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:07.401130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:07.401130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:25:07.401130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:25:07.401130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:25:07.402130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:25:07.402130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:07.402130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:07.402130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:07.402130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:07.402130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:07.402130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:07.402130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:07.402130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:07.402130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:07.402130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:07.402130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:07.402130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:07.402130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:07.402130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:07.402130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:07.402130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:07.403041 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:25:07.403041 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:25:07.403041 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:07.403041 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:07.403041 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 418/838] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:07.408067 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21adba1-3c89-407a-95c5-5e1d70777fdb None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:07.437249 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:07.479609 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 421/839] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:07 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:25:07.481535 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:07.499972 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4c6c1a-25d7-4124-9e36-8ea423b374e4 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:07.533786 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21adba1-3c89-407a-95c5-5e1d70777fdb None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:07.574681 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343d5d70-78aa-4158-839f-19d58d42544f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:07.575348 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d5d70-78aa-4158-839f-19d58d42544f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:07.575670 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d5d70-78aa-4158-839f-19d58d42544f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:07.575946 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d5d70-78aa-4158-839f-19d58d42544f None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:07.577568 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343d5d70-78aa-4158-839f-19d58d42544f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:07.577873 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343d5d70-78aa-4158-839f-19d58d42544f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:07.579247 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343d5d70-78aa-4158-839f-19d58d42544f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:07.597522 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 419/840] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:25:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:07.598897 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343d5d70-78aa-4158-839f-19d58d42544f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:07.626400 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4c6c1a-25d7-4124-9e36-8ea423b374e4 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:07.668326 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce43c541-1bd7-4878-b1f6-1c41db5c7585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:07.669763 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce43c541-1bd7-4878-b1f6-1c41db5c7585 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:07.670251 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce43c541-1bd7-4878-b1f6-1c41db5c7585 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:07.670728 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce43c541-1bd7-4878-b1f6-1c41db5c7585 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:07.671743 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce43c541-1bd7-4878-b1f6-1c41db5c7585 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:07.673486 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce43c541-1bd7-4878-b1f6-1c41db5c7585 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:07.688852 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-ce43c541-1bd7-4878-b1f6-1c41db5c7585 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 16:25:07.688852 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:07.688852 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:25:07.688852 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:25:07.688852 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:25:07.688852 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:25:07.688852 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:25:07.688852 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:07.688852 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 16:25:07.688852 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:07.688852 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:25:07.688852 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:07.688852 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:07.688852 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:07.688852 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:07.688852 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:07.688852 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:07.689907 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:07.689907 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:07.689907 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:07.689907 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:07.689907 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:07.689907 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:07.689907 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:25:07.689907 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:25:07.689907 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:25:07.689907 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:25:07.689907 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:07.689907 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:07.689907 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:07.689907 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:07.689907 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:07.689907 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:07.690734 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:07.690734 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:07.690734 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:07.690734 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:07.690734 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:07.690734 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:07.690734 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:07.690734 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:07.690734 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:07.690734 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:07.690734 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:25:07.690734 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:07.690734 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 16:25:07.690734 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:07.699357 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 422/841] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 18 16:25:07 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:07.700674 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce43c541-1bd7-4878-b1f6-1c41db5c7585 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:07.712600 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343d5d70-78aa-4158-839f-19d58d42544f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:07.738237 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:07.752949 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:07.757485 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5ff388-8e04-4a95-a8f0-5e7246d7b2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:07.758325 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ff388-8e04-4a95-a8f0-5e7246d7b2a9 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:07.758630 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ff388-8e04-4a95-a8f0-5e7246d7b2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:07.758902 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ff388-8e04-4a95-a8f0-5e7246d7b2a9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:07.767287 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5ff388-8e04-4a95-a8f0-5e7246d7b2a9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:07.768762 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5ff388-8e04-4a95-a8f0-5e7246d7b2a9 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:07.789815 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-0f5ff388-8e04-4a95-a8f0-5e7246d7b2a9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 16:25:07.789815 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:07.789815 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:07.789815 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:07.789815 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:07.789815 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:07.789815 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:07.789815 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:07.789815 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:07.789815 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:07.789815 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:07.789815 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:07.789815 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:25:07.789815 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:25:07.789815 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:25:07.789815 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:25:07.791460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:07.791460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:07.791460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:25:07.791460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:07.791460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:25:07.791460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:07.791460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:07.791460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:07.791460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:07.791460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:07.791460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:07.791460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:07.791460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:07.791460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:07.791460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:07.791460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:07.791460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:07.791460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:07.792449 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:07.792449 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:07.792449 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:25:07.792449 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:25:07.792449 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:25:07.792449 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:25:07.792449 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:25:07.792449 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:25:07.792449 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 16:25:07.792449 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:07.792449 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 420/842] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 18 16:25:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:25:07.793538 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d6e9db-ea33-4a9f-9f03-67ff691b6f7b None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:07.793701 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d6e9db-ea33-4a9f-9f03-67ff691b6f7b None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:07.794186 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d6e9db-ea33-4a9f-9f03-67ff691b6f7b None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:07.795792 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 421/843] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:25:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:07.799728 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d6e9db-ea33-4a9f-9f03-67ff691b6f7b None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:07.826190 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce43c541-1bd7-4878-b1f6-1c41db5c7585 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:07.873408 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef47149e-f083-4bf5-ad0a-90f02909f3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:07.874397 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef47149e-f083-4bf5-ad0a-90f02909f3df None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:07.874687 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef47149e-f083-4bf5-ad0a-90f02909f3df None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:07.875215 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef47149e-f083-4bf5-ad0a-90f02909f3df None admin] PATH_INFO: `/v3/services` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:07.876407 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef47149e-f083-4bf5-ad0a-90f02909f3df None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:07.876864 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef47149e-f083-4bf5-ad0a-90f02909f3df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:07.878521 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef47149e-f083-4bf5-ad0a-90f02909f3df None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:07.887993 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 423/844] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 18 16:25:07 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:07.887993 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c4919-924f-4bda-9d28-6006d8f063ee None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:07.887993 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c4919-924f-4bda-9d28-6006d8f063ee None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:07.887993 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c4919-924f-4bda-9d28-6006d8f063ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:07.902555 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-f40c4919-924f-4bda-9d28-6006d8f063ee None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:07.904026 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-f40c4919-924f-4bda-9d28-6006d8f063ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:07.955312 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d6e9db-ea33-4a9f-9f03-67ff691b6f7b None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:08.001364 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cceccca-2c2d-4d64-ac5c-8031c307204d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:08.002214 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cceccca-2c2d-4d64-ac5c-8031c307204d None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:08.002502 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cceccca-2c2d-4d64-ac5c-8031c307204d None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:08.002767 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cceccca-2c2d-4d64-ac5c-8031c307204d None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:08.003654 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cceccca-2c2d-4d64-ac5c-8031c307204d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:08.003966 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cceccca-2c2d-4d64-ac5c-8031c307204d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:08.005433 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cceccca-2c2d-4d64-ac5c-8031c307204d None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:08.011606 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 424/845] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:07 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:08.011606 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40c4919-924f-4bda-9d28-6006d8f063ee None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:08.028758 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 422/846] 10.222.0.26 () {56 vars in 1135 bytes} [Mon Jul 18 16:25:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:08.036819 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cceccca-2c2d-4d64-ac5c-8031c307204d None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:08.058777 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:08.079883 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:08.159259 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cceccca-2c2d-4d64-ac5c-8031c307204d None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:08.160036 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40c4919-924f-4bda-9d28-6006d8f063ee None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:08.202479 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d6deef-7f9b-42eb-8917-3286a94f201a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:08.203125 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d6deef-7f9b-42eb-8917-3286a94f201a None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:08.203404 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d6deef-7f9b-42eb-8917-3286a94f201a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:08.203736 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d6deef-7f9b-42eb-8917-3286a94f201a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:08.206567 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f0ee82-a249-4009-812b-41950c911e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:08.207242 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0ee82-a249-4009-812b-41950c911e4d None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:08.207585 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0ee82-a249-4009-812b-41950c911e4d None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:08.207841 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0ee82-a249-4009-812b-41950c911e4d None admin] PATH_INFO: `/v3/services` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:08.208565 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f0ee82-a249-4009-812b-41950c911e4d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:08.208870 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f0ee82-a249-4009-812b-41950c911e4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:08.210264 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f0ee82-a249-4009-812b-41950c911e4d None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:08.215950 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 425/847] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 18 16:25:08 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:08.216599 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f0ee82-a249-4009-812b-41950c911e4d None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:08.323139 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d6deef-7f9b-42eb-8917-3286a94f201a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:08.324807 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d6deef-7f9b-42eb-8917-3286a94f201a None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:08.366164 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:08.370303 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:08.377161 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f0ee82-a249-4009-812b-41950c911e4d None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:08.431943 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a546b6-2324-44e1-bb43-5a19d2a787bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:08.432854 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a546b6-2324-44e1-bb43-5a19d2a787bf None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:08.433212 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a546b6-2324-44e1-bb43-5a19d2a787bf None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:08.433558 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a546b6-2324-44e1-bb43-5a19d2a787bf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:08.448097 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a546b6-2324-44e1-bb43-5a19d2a787bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:25:08.455803 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 423/848] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:25:08 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 16:25:08.457786 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-e7a546b6-2324-44e1-bb43-5a19d2a787bf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:25:08.457786 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:08.457786 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:08.457786 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:08.457786 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:08.457786 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:08.457786 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:08.457786 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:08.457786 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:08.457786 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:08.457786 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:08.457786 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:08.457786 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:25:08.457786 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:25:08.457786 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:25:08.457786 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:25:08.458683 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:25:08.458683 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:25:08.458683 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:25:08.458683 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:25:08.458683 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:08.458683 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:08.458683 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:08.458683 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:08.458683 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:08.458683 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:08.458683 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:08.458683 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:08.458683 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:08.458683 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:08.458683 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:08.458683 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:08.459556 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:08.459556 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:08.459556 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:08.459556 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:08.459556 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:25:08.459556 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:25:08.459556 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:25:08.459556 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:25:08.459556 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:25:08.459556 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:25:08.459556 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:25:08.459556 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:08.459556 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 426/849] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 18 16:25:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:08.461583 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d6deef-7f9b-42eb-8917-3286a94f201a None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:08.466855 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a546b6-2324-44e1-bb43-5a19d2a787bf None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:08.598779 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d6deef-7f9b-42eb-8917-3286a94f201a None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:08.598992 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a546b6-2324-44e1-bb43-5a19d2a787bf None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:08.647303 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be8437c-e567-4629-902d-06db97fad139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:08.648264 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be8437c-e567-4629-902d-06db97fad139 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:08.648737 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be8437c-e567-4629-902d-06db97fad139 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:08.649166 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be8437c-e567-4629-902d-06db97fad139 None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:08.650092 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f829e5e0-5755-4334-b76e-87e475e27b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:08.650585 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be8437c-e567-4629-902d-06db97fad139 None admin] RBAC: Adding query filter params (name=service) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:08.650923 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f829e5e0-5755-4334-b76e-87e475e27b9f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:08.651221 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f829e5e0-5755-4334-b76e-87e475e27b9f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:08.651364 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be8437c-e567-4629-902d-06db97fad139 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:08.651510 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f829e5e0-5755-4334-b76e-87e475e27b9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:08.653056 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be8437c-e567-4629-902d-06db97fad139 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:08.655422 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:08.669286 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 427/850] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 18 16:25:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:08.669491 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f829e5e0-5755-4334-b76e-87e475e27b9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:08.669491 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f829e5e0-5755-4334-b76e-87e475e27b9f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:08.671717 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be8437c-e567-4629-902d-06db97fad139 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:08.692427 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-f829e5e0-5755-4334-b76e-87e475e27b9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:08.692427 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:08.692427 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:08.692427 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:08.692427 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:08.692427 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:08.692427 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:08.692427 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:08.692427 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:08.692427 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:08.692427 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:08.692427 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:08.692427 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:25:08.692427 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:25:08.692427 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:25:08.693584 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:25:08.693584 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:08.693584 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:08.693584 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:08.693584 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:08.693584 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:08.693584 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:08.693584 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:08.693584 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:08.693584 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:08.693584 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:08.693584 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:08.693584 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:08.693584 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:08.693584 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:08.693584 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:08.693584 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:08.694691 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:25:08.694691 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:25:08.694691 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:08.694691 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:08.695591 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:08.698856 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 424/851] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:25:08.707026 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f829e5e0-5755-4334-b76e-87e475e27b9f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:08.834287 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f829e5e0-5755-4334-b76e-87e475e27b9f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:08.870716 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be8437c-e567-4629-902d-06db97fad139 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:08.877325 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c293637-746a-4386-8185-173adebfe4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:08.877998 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c293637-746a-4386-8185-173adebfe4c2 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:08.878281 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c293637-746a-4386-8185-173adebfe4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:08.878567 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c293637-746a-4386-8185-173adebfe4c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:08.879272 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c293637-746a-4386-8185-173adebfe4c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:08.879622 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c293637-746a-4386-8185-173adebfe4c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:08.880994 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c293637-746a-4386-8185-173adebfe4c2 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:08.899748 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 425/852] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:25:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:08.903300 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c293637-746a-4386-8185-173adebfe4c2 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:08.938351 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39539eb8-f420-43ee-8c3c-e17b52df5ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:08.940435 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39539eb8-f420-43ee-8c3c-e17b52df5ed2 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:08.941260 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39539eb8-f420-43ee-8c3c-e17b52df5ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:08.941800 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39539eb8-f420-43ee-8c3c-e17b52df5ed2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:08.959845 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39539eb8-f420-43ee-8c3c-e17b52df5ed2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:08.963102 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39539eb8-f420-43ee-8c3c-e17b52df5ed2 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:08.964540 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:08.992270 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-39539eb8-f420-43ee-8c3c-e17b52df5ed2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:08.992270 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:08.992270 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:08.992270 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:08.992270 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:08.992270 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:08.992270 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:08.992270 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:08.992270 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:08.992270 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:08.992270 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:08.992270 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:08.992270 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:25:08.992270 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:25:08.992270 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:25:08.993407 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:25:08.993407 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:08.993407 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:08.993407 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:08.993407 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:08.993407 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:08.993407 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:08.993407 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:08.993407 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:08.993407 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:08.993407 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:08.993407 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:08.993407 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:08.993407 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:08.993407 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:08.993407 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:08.993407 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:08.994614 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:25:08.994614 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:25:08.994614 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:08.994614 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:08.994614 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 428/853] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:25:08.998931 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39539eb8-f420-43ee-8c3c-e17b52df5ed2 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:09.041151 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c293637-746a-4386-8185-173adebfe4c2 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:09.082505 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d2cb0b-e2ff-4442-a352-2837ef625cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:09.083462 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d2cb0b-e2ff-4442-a352-2837ef625cc8 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:09.083827 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d2cb0b-e2ff-4442-a352-2837ef625cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:09.084167 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d2cb0b-e2ff-4442-a352-2837ef625cc8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:09.088171 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:09.092121 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d2cb0b-e2ff-4442-a352-2837ef625cc8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:09.093651 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d2cb0b-e2ff-4442-a352-2837ef625cc8 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:09.124628 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-64d2cb0b-e2ff-4442-a352-2837ef625cc8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:25:09.124628 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:09.124628 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:09.124628 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:09.124628 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:09.124628 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:09.124628 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:09.124628 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:09.124628 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:09.124628 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:09.124628 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:09.124628 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:09.124628 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:25:09.124628 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:25:09.124628 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:25:09.125551 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:25:09.125551 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:09.125551 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:09.125551 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:09.125551 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:09.125551 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:09.125551 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:09.125551 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:09.125551 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:09.125551 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:09.125551 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:09.125551 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:09.125551 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:09.125551 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:09.125551 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:09.125551 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:09.125551 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:09.126568 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:25:09.126568 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:25:09.126568 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:25:09.126568 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:25:09.126568 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:25:09.126568 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:25:09.126568 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:25:09.126568 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:09.126568 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 426/854] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:25:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:09.126568 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d2cb0b-e2ff-4442-a352-2837ef625cc8 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:09.130417 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39539eb8-f420-43ee-8c3c-e17b52df5ed2 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:09.168371 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029c22d5-18c3-49ac-b4f9-7400789f82b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:09.169394 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029c22d5-18c3-49ac-b4f9-7400789f82b1 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:09.181806 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029c22d5-18c3-49ac-b4f9-7400789f82b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:09.181806 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029c22d5-18c3-49ac-b4f9-7400789f82b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:09.181806 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029c22d5-18c3-49ac-b4f9-7400789f82b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:09.181806 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029c22d5-18c3-49ac-b4f9-7400789f82b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:09.181806 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029c22d5-18c3-49ac-b4f9-7400789f82b1 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:09.189089 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 429/855] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:25:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:09.198513 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029c22d5-18c3-49ac-b4f9-7400789f82b1 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:09.259192 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d2cb0b-e2ff-4442-a352-2837ef625cc8 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:09.269481 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:09.298762 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c011edfd-1767-4979-8b1d-0916ce316060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:09.299443 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011edfd-1767-4979-8b1d-0916ce316060 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:09.299751 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011edfd-1767-4979-8b1d-0916ce316060 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:09.300033 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011edfd-1767-4979-8b1d-0916ce316060 None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:09.300834 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c011edfd-1767-4979-8b1d-0916ce316060 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:09.301145 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c011edfd-1767-4979-8b1d-0916ce316060 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:09.302588 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c011edfd-1767-4979-8b1d-0916ce316060 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:09.320912 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 427/856] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 18 16:25:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:09.320912 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3441134d-79e1-423c-b9dd-6aebbe43b1b3 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:09.320912 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3441134d-79e1-423c-b9dd-6aebbe43b1b3 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:09.320912 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3441134d-79e1-423c-b9dd-6aebbe43b1b3 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:09.320912 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 428/857] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:25:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:09.322895 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3441134d-79e1-423c-b9dd-6aebbe43b1b3 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:09.329834 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029c22d5-18c3-49ac-b4f9-7400789f82b1 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:09.378286 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:09.385084 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4d860a-52e7-4f4f-a2c2-a40b5e23cf15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:09.387072 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d860a-52e7-4f4f-a2c2-a40b5e23cf15 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:09.391229 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d860a-52e7-4f4f-a2c2-a40b5e23cf15 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:09.391229 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d860a-52e7-4f4f-a2c2-a40b5e23cf15 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:09.396124 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4d860a-52e7-4f4f-a2c2-a40b5e23cf15 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:09.397813 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4d860a-52e7-4f4f-a2c2-a40b5e23cf15 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:09.415023 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-fc4d860a-52e7-4f4f-a2c2-a40b5e23cf15 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 16:25:09.415023 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:09.415023 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:09.415023 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:09.415023 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:09.415023 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:09.415023 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:09.415023 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:09.415023 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:09.415023 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:09.415023 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:09.415023 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:09.415023 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:25:09.415023 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:25:09.415023 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:25:09.415023 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:25:09.417530 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:09.417530 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:09.417530 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:25:09.417530 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:09.417530 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:25:09.417530 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:09.417530 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:09.417530 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:09.417530 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:09.417530 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:09.417530 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:09.417530 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:09.417530 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:09.417530 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:09.417530 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:09.417530 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:09.417530 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:09.417530 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:09.418497 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:09.418497 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:09.418497 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:25:09.418497 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:25:09.418497 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:25:09.418497 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:25:09.418497 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:25:09.418497 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:25:09.418497 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 16:25:09.418497 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:09.418497 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 430/858] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Jul 18 16:25:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:09.419938 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ce6ae-5813-49d9-acb0-741f21491ba5 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:09.419938 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ce6ae-5813-49d9-acb0-741f21491ba5 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:09.420268 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ce6ae-5813-49d9-acb0-741f21491ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:09.441577 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-493ce6ae-5813-49d9-acb0-741f21491ba5 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:09.442825 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-493ce6ae-5813-49d9-acb0-741f21491ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:09.450914 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3441134d-79e1-423c-b9dd-6aebbe43b1b3 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:09.492145 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a98ffc-11ab-49b4-86a8-2a2e3101fa81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:09.492855 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a98ffc-11ab-49b4-86a8-2a2e3101fa81 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:09.493164 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a98ffc-11ab-49b4-86a8-2a2e3101fa81 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:09.493997 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a98ffc-11ab-49b4-86a8-2a2e3101fa81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:09.494934 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a98ffc-11ab-49b4-86a8-2a2e3101fa81 None admin] RBAC: Adding query filter params (role.id=98eaadb3669f4b608d572c001742f559, scope.project.id=eb9747e2299c4e0991fd7edb4462c7e5, user.id=43854d7a6df2466eb7976fabe6011767) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:09.495501 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a98ffc-11ab-49b4-86a8-2a2e3101fa81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:09.497078 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a98ffc-11ab-49b4-86a8-2a2e3101fa81 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:09.506057 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 429/859] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 18 16:25:09 2022] GET /identity/v3/role_assignments?user.id=43854d7a6df2466eb7976fabe6011767&scope.project.id=eb9747e2299c4e0991fd7edb4462c7e5&role.id=98eaadb3669f4b608d572c001742f559 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:09.506690 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a98ffc-11ab-49b4-86a8-2a2e3101fa81 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:09.555188 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 431/860] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:09 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:09.567839 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:09.580328 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493ce6ae-5813-49d9-acb0-741f21491ba5 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:09.668264 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a98ffc-11ab-49b4-86a8-2a2e3101fa81 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:09.700908 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493ce6ae-5813-49d9-acb0-741f21491ba5 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:09.717107 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f2109-875a-459e-a3c7-74fe1d8f168a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:09.718182 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f2109-875a-459e-a3c7-74fe1d8f168a None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:09.718607 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f2109-875a-459e-a3c7-74fe1d8f168a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:09.719069 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f2109-875a-459e-a3c7-74fe1d8f168a None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:09.721126 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3f2109-875a-459e-a3c7-74fe1d8f168a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:09.721541 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3f2109-875a-459e-a3c7-74fe1d8f168a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:09.723346 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3f2109-875a-459e-a3c7-74fe1d8f168a None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:09.730183 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:09.749804 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c89d746-b6b0-4bc8-a601-7842b1aa7353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:09.750630 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c89d746-b6b0-4bc8-a601-7842b1aa7353 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:09.750913 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c89d746-b6b0-4bc8-a601-7842b1aa7353 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:09.751185 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c89d746-b6b0-4bc8-a601-7842b1aa7353 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:09.751585 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 430/861] 10.222.0.26 () {56 vars in 1149 bytes} [Mon Jul 18 16:25:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:09.751988 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c89d746-b6b0-4bc8-a601-7842b1aa7353 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:09.753451 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c89d746-b6b0-4bc8-a601-7842b1aa7353 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:09.757079 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f2109-875a-459e-a3c7-74fe1d8f168a None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:09.767877 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-2c89d746-b6b0-4bc8-a601-7842b1aa7353 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 16:25:09.767877 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:09.767877 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:25:09.767877 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:25:09.767877 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:25:09.767877 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:25:09.767877 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:25:09.767877 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:09.767877 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 16:25:09.767877 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:09.767877 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:25:09.767877 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:09.767877 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:09.767877 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:09.767877 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:09.767877 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:09.769054 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:09.769054 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:09.769054 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:09.769054 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:09.769054 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:09.769054 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:09.769054 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:09.769054 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:25:09.769054 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:25:09.769054 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:25:09.769054 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:25:09.769054 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:09.769054 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:09.769054 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:09.769054 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:09.769054 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:09.769918 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:09.769918 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:09.769918 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:09.769918 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:09.769918 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:09.769918 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:09.769918 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:09.769918 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:09.769918 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:09.769918 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:09.769918 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:09.769918 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:25:09.769918 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:09.769918 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 16:25:09.769918 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:09.769918 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 432/862] 10.222.0.26 () {56 vars in 1132 bytes} [Mon Jul 18 16:25:09 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:25:09.775841 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c89d746-b6b0-4bc8-a601-7842b1aa7353 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:09.882978 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:09.896009 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f2109-875a-459e-a3c7-74fe1d8f168a None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:09.935521 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c89d746-b6b0-4bc8-a601-7842b1aa7353 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:09.942620 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751eab8e-4fdd-4083-a419-3fee6b3777b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:09.943597 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751eab8e-4fdd-4083-a419-3fee6b3777b4 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:09.944063 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751eab8e-4fdd-4083-a419-3fee6b3777b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:09.945036 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751eab8e-4fdd-4083-a419-3fee6b3777b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:09.981623 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbbfbe6-e90c-4622-ba4d-3f529e419e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:09.982415 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbfbe6-e90c-4622-ba4d-3f529e419e6f None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:09.982954 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbfbe6-e90c-4622-ba4d-3f529e419e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:09.983304 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbfbe6-e90c-4622-ba4d-3f529e419e6f None admin] PATH_INFO: `/v3/services` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:09.984493 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbbfbe6-e90c-4622-ba4d-3f529e419e6f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:09.984873 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbbfbe6-e90c-4622-ba4d-3f529e419e6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:09.986675 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbbfbe6-e90c-4622-ba4d-3f529e419e6f None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:09.995552 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 433/863] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Jul 18 16:25:09 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:10.004433 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbbfbe6-e90c-4622-ba4d-3f529e419e6f None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:10.047171 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751eab8e-4fdd-4083-a419-3fee6b3777b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:10.048949 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751eab8e-4fdd-4083-a419-3fee6b3777b4 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:10.075192 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:10.133419 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbbfbe6-e90c-4622-ba4d-3f529e419e6f None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:10.172467 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:10.174907 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f044e61-409f-4709-b270-b08f04ac9fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:10.176074 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f044e61-409f-4709-b270-b08f04ac9fd4 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:10.176589 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f044e61-409f-4709-b270-b08f04ac9fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:10.177068 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f044e61-409f-4709-b270-b08f04ac9fd4 None admin] PATH_INFO: `/v3/services` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:10.177987 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f044e61-409f-4709-b270-b08f04ac9fd4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:10.178574 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f044e61-409f-4709-b270-b08f04ac9fd4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:10.185376 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f044e61-409f-4709-b270-b08f04ac9fd4 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:10.185608 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 431/864] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:25:09 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 18 16:25:10.187153 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 434/865] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Jul 18 16:25:10 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:10.192733 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751eab8e-4fdd-4083-a419-3fee6b3777b4 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:10.321205 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751eab8e-4fdd-4083-a419-3fee6b3777b4 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:10.362254 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72e7683-881b-41d2-8251-da8057073167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:10.363450 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e7683-881b-41d2-8251-da8057073167 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:10.363970 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e7683-881b-41d2-8251-da8057073167 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:10.364453 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e7683-881b-41d2-8251-da8057073167 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:10.367830 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41825be-ce4e-421f-b10e-0f3572ca2fd8 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:10.368701 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41825be-ce4e-421f-b10e-0f3572ca2fd8 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:10.368990 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41825be-ce4e-421f-b10e-0f3572ca2fd8 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:10.370386 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 435/866] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:25:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:10.376063 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2192cc92-8ecf-4e01-a4f5-1d55f1e0239b None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:10.376339 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2192cc92-8ecf-4e01-a4f5-1d55f1e0239b None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:10.376623 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2192cc92-8ecf-4e01-a4f5-1d55f1e0239b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:10.379837 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72e7683-881b-41d2-8251-da8057073167 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:10.381559 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72e7683-881b-41d2-8251-da8057073167 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:10.400983 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-b72e7683-881b-41d2-8251-da8057073167 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:10.400983 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:10.400983 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:10.400983 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:10.400983 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:10.400983 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:10.400983 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:10.400983 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:10.400983 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:10.400983 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:10.400983 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:10.400983 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:10.400983 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:25:10.400983 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:25:10.400983 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:25:10.401957 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:25:10.401957 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:10.401957 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:10.401957 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:10.401957 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:10.401957 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:10.401957 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:10.401957 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:10.401957 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:10.401957 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:10.401957 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:10.401957 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:10.401957 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:10.401957 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:10.401957 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:10.401957 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:10.401957 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:10.402839 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:25:10.402839 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:25:10.402839 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:10.402839 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:10.405491 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 432/867] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:25:10.410809 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-2192cc92-8ecf-4e01-a4f5-1d55f1e0239b None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:10.411442 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72e7683-881b-41d2-8251-da8057073167 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:10.412123 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-2192cc92-8ecf-4e01-a4f5-1d55f1e0239b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:10.476832 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:10.516027 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 436/868] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:10 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:10.539401 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2192cc92-8ecf-4e01-a4f5-1d55f1e0239b None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:10.560949 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72e7683-881b-41d2-8251-da8057073167 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:10.596996 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:10.608924 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ed6cba-2e5c-4ce7-b103-16181862ef48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:10.609802 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed6cba-2e5c-4ce7-b103-16181862ef48 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:10.610200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed6cba-2e5c-4ce7-b103-16181862ef48 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:10.611020 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed6cba-2e5c-4ce7-b103-16181862ef48 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:10.611883 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ed6cba-2e5c-4ce7-b103-16181862ef48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:10.612564 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ed6cba-2e5c-4ce7-b103-16181862ef48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:10.614202 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ed6cba-2e5c-4ce7-b103-16181862ef48 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:10.641270 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 433/869] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:25:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:10.644347 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ed6cba-2e5c-4ce7-b103-16181862ef48 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:10.697842 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2192cc92-8ecf-4e01-a4f5-1d55f1e0239b None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:10.757986 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473de236-284d-434c-a738-125c5f7008fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:10.759453 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473de236-284d-434c-a738-125c5f7008fe None admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:10.759921 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473de236-284d-434c-a738-125c5f7008fe None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:10.760379 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473de236-284d-434c-a738-125c5f7008fe None admin] PATH_INFO: `/v3/services` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:10.761279 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473de236-284d-434c-a738-125c5f7008fe None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:10.762870 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473de236-284d-434c-a738-125c5f7008fe None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:10.808637 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ed6cba-2e5c-4ce7-b103-16181862ef48 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:10.811352 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:10.867536 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61982546-1fa1-4a07-b4d2-ab2a052fa94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:10.867536 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61982546-1fa1-4a07-b4d2-ab2a052fa94c None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:10.867536 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61982546-1fa1-4a07-b4d2-ab2a052fa94c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:10.867536 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61982546-1fa1-4a07-b4d2-ab2a052fa94c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:10.872918 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 437/870] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jul 18 16:25:10 2022] POST /identity/v3/services => generated 251 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:25:10.879080 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61982546-1fa1-4a07-b4d2-ab2a052fa94c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:10.881545 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61982546-1fa1-4a07-b4d2-ab2a052fa94c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:10.903197 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-61982546-1fa1-4a07-b4d2-ab2a052fa94c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:25:10.903197 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:10.903197 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:10.903197 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:10.903197 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:10.903197 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:10.903197 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:10.903197 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:10.903197 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:10.903197 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:10.903197 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:10.903197 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:10.903197 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:25:10.903197 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:25:10.903197 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:25:10.904836 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:25:10.904836 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:10.904836 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:10.904836 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:10.904836 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:10.904836 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:10.904836 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:10.904836 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:10.904836 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:10.904836 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:10.904836 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:10.904836 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:10.904836 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:10.904836 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:10.904836 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:10.904836 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:10.904836 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:10.905819 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:25:10.905819 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:25:10.905819 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:25:10.905819 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:25:10.905819 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:25:10.905819 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:25:10.905819 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:25:10.905819 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:10.909775 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 434/871] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:25:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:10.916202 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473de236-284d-434c-a738-125c5f7008fe None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:11.053262 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:11.060131 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473de236-284d-434c-a738-125c5f7008fe None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:11.099706 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3898fe8e-60f6-42cc-95a5-da10478e60d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:11.100382 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3898fe8e-60f6-42cc-95a5-da10478e60d2 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:11.100663 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3898fe8e-60f6-42cc-95a5-da10478e60d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:11.100964 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3898fe8e-60f6-42cc-95a5-da10478e60d2 None admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:11.102294 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3898fe8e-60f6-42cc-95a5-da10478e60d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:11.102686 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3898fe8e-60f6-42cc-95a5-da10478e60d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:11.104630 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3898fe8e-60f6-42cc-95a5-da10478e60d2 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:11.118993 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 438/872] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 18 16:25:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:11.125768 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61982546-1fa1-4a07-b4d2-ab2a052fa94c None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:11.256149 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61982546-1fa1-4a07-b4d2-ab2a052fa94c None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:11.294889 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b8e21-5394-46ad-a199-875285f58d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:11.295582 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b8e21-5394-46ad-a199-875285f58d62 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:11.295865 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b8e21-5394-46ad-a199-875285f58d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:11.296136 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b8e21-5394-46ad-a199-875285f58d62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:11.296969 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99b8e21-5394-46ad-a199-875285f58d62 None admin] RBAC: Adding query filter params (role.id=4fe970e98f394bbfb096c4879bebdf97, scope.project.id=eb9747e2299c4e0991fd7edb4462c7e5, user.id=da799b0044f54da6835aa06e43ce1a6f) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:11.297299 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99b8e21-5394-46ad-a199-875285f58d62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:11.298740 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99b8e21-5394-46ad-a199-875285f58d62 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:11.306200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 435/873] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 18 16:25:11 2022] GET /identity/v3/role_assignments?user.id=da799b0044f54da6835aa06e43ce1a6f&scope.project.id=eb9747e2299c4e0991fd7edb4462c7e5&role.id=4fe970e98f394bbfb096c4879bebdf97 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:11.724324 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4362f8-4954-4df2-be8e-dede408d5422 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:11.724842 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4362f8-4954-4df2-be8e-dede408d5422 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:11.724842 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4362f8-4954-4df2-be8e-dede408d5422 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:11.726456 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 439/874] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:25:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:11.732540 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693af5c7-7530-4bd0-a919-bdc52628dc3b None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:11.732822 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693af5c7-7530-4bd0-a919-bdc52628dc3b None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:11.733104 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693af5c7-7530-4bd0-a919-bdc52628dc3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:11.755200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-693af5c7-7530-4bd0-a919-bdc52628dc3b None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:11.757024 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-693af5c7-7530-4bd0-a919-bdc52628dc3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:11.784203 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:11.845458 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 436/875] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:11 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:25:11.858688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4362f8-4954-4df2-be8e-dede408d5422 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:11.975862 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4362f8-4954-4df2-be8e-dede408d5422 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:12.015912 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852f7c2e-dfb5-4947-aaeb-0d08578687d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:12.016735 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f7c2e-dfb5-4947-aaeb-0d08578687d3 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:12.017017 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f7c2e-dfb5-4947-aaeb-0d08578687d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:12.017294 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f7c2e-dfb5-4947-aaeb-0d08578687d3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:12.017952 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852f7c2e-dfb5-4947-aaeb-0d08578687d3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:12.019326 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852f7c2e-dfb5-4947-aaeb-0d08578687d3 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:12.033482 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-852f7c2e-dfb5-4947-aaeb-0d08578687d3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 16:25:12.033482 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:12.033482 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:25:12.033482 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:25:12.033482 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:25:12.033482 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:25:12.033482 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:25:12.033482 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:12.033482 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 16:25:12.033482 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:12.033482 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:25:12.033482 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:12.033482 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:12.033482 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:12.033482 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:12.033482 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:12.033482 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:12.034570 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:12.034570 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:12.034570 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:12.034570 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:12.034570 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:12.034570 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:12.034570 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:25:12.034570 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:25:12.034570 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:25:12.034570 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:25:12.034570 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:12.034570 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:12.034570 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:12.034570 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:12.034570 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:12.034570 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:12.035439 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:12.035439 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:12.035439 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:12.035439 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:12.035439 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:12.035439 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:12.035439 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:12.035439 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:12.035439 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:12.035439 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:12.035439 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:25:12.035439 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:12.035439 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 16:25:12.035439 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:12.039597 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 440/876] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Jul 18 16:25:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:12.042334 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693af5c7-7530-4bd0-a919-bdc52628dc3b None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:12.158734 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693af5c7-7530-4bd0-a919-bdc52628dc3b None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:12.202055 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c1cd0d-be19-4efd-ad35-46931913418b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:12.202740 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c1cd0d-be19-4efd-ad35-46931913418b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:12.203023 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c1cd0d-be19-4efd-ad35-46931913418b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:12.203295 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c1cd0d-be19-4efd-ad35-46931913418b None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:12.204052 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c1cd0d-be19-4efd-ad35-46931913418b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:12.204355 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c1cd0d-be19-4efd-ad35-46931913418b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:12.206230 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c1cd0d-be19-4efd-ad35-46931913418b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:12.213203 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 437/877] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Jul 18 16:25:12 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:12.219028 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852f7c2e-dfb5-4947-aaeb-0d08578687d3 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:12.248687 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:12.322805 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae8f8a-be4d-43e1-b6ed-7ae08dd47b00 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:12.323146 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae8f8a-be4d-43e1-b6ed-7ae08dd47b00 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:12.323370 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae8f8a-be4d-43e1-b6ed-7ae08dd47b00 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:12.325042 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 438/878] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:25:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:12.330647 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c616efb-0e6d-4b12-8ad7-5de9524dc597 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:12.330826 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c616efb-0e6d-4b12-8ad7-5de9524dc597 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:12.331053 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c616efb-0e6d-4b12-8ad7-5de9524dc597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:12.337392 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852f7c2e-dfb5-4947-aaeb-0d08578687d3 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:12.348334 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-2c616efb-0e6d-4b12-8ad7-5de9524dc597 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:12.350345 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-2c616efb-0e6d-4b12-8ad7-5de9524dc597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:12.372247 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:12.375868 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e457d564-e6eb-426a-8fe6-00304b483ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:12.376544 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457d564-e6eb-426a-8fe6-00304b483ba9 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:12.376826 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457d564-e6eb-426a-8fe6-00304b483ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:12.377208 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457d564-e6eb-426a-8fe6-00304b483ba9 None admin] PATH_INFO: `/v3/services` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:12.378298 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e457d564-e6eb-426a-8fe6-00304b483ba9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:12.378686 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e457d564-e6eb-426a-8fe6-00304b483ba9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:12.380105 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e457d564-e6eb-426a-8fe6-00304b483ba9 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:12.385793 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 441/879] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Jul 18 16:25:12 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:12.445529 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 439/880] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:12 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:25:12.470422 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e457d564-e6eb-426a-8fe6-00304b483ba9 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:12.569646 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:12.588093 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e457d564-e6eb-426a-8fe6-00304b483ba9 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:12.626358 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb14ba0-1b3a-4ab5-84b2-4bafca73727c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:12.627044 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb14ba0-1b3a-4ab5-84b2-4bafca73727c None admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:12.627708 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb14ba0-1b3a-4ab5-84b2-4bafca73727c None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:12.628417 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb14ba0-1b3a-4ab5-84b2-4bafca73727c None admin] PATH_INFO: `/v3/services` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:12.629172 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb14ba0-1b3a-4ab5-84b2-4bafca73727c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:12.630556 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb14ba0-1b3a-4ab5-84b2-4bafca73727c None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:12.753095 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 442/881] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jul 18 16:25:12 2022] POST /identity/v3/services => generated 269 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:25:13.203398 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d68e70-a3e8-406b-b8c9-6510861324e3 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:13.203937 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d68e70-a3e8-406b-b8c9-6510861324e3 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:13.203937 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d68e70-a3e8-406b-b8c9-6510861324e3 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:13.205589 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 440/882] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:25:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:13.211237 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f562a5-527b-4350-af66-12c043c777d2 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:13.212159 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f562a5-527b-4350-af66-12c043c777d2 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:13.212447 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f562a5-527b-4350-af66-12c043c777d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:13.231020 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-f4f562a5-527b-4350-af66-12c043c777d2 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:13.232361 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-f4f562a5-527b-4350-af66-12c043c777d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:13.326532 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 443/883] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:13 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:13.352280 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d68e70-a3e8-406b-b8c9-6510861324e3 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:13.468172 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d68e70-a3e8-406b-b8c9-6510861324e3 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:13.505970 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb252b1c-b4a3-48aa-8594-76beb3edae8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:13.506768 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb252b1c-b4a3-48aa-8594-76beb3edae8a None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:13.507049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb252b1c-b4a3-48aa-8594-76beb3edae8a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:13.507320 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb252b1c-b4a3-48aa-8594-76beb3edae8a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:13.508000 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb252b1c-b4a3-48aa-8594-76beb3edae8a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:13.510595 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb252b1c-b4a3-48aa-8594-76beb3edae8a None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:13.523229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-fb252b1c-b4a3-48aa-8594-76beb3edae8a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 16:25:13.523229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:13.523229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:25:13.523229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:25:13.523229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:25:13.523229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:25:13.523229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:25:13.523229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:13.523229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 16:25:13.523229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:13.523229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:25:13.523229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:13.523229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:13.523229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:13.523229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:13.523229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:13.523229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:13.524354 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:13.524354 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:13.524354 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:13.524354 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:13.524354 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:13.524354 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:13.524354 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:25:13.524354 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:25:13.524354 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:25:13.524354 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:25:13.524354 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:13.524354 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:13.524354 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:13.524354 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:13.524354 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:13.524354 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:13.525686 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:13.525686 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:13.525686 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:13.525686 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:13.525686 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:13.525686 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:13.525686 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:13.525686 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:13.525686 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:13.525686 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:13.525686 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:25:13.525686 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:13.525686 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 16:25:13.525686 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:13.525686 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 441/884] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 18 16:25:13 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:13.531048 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f562a5-527b-4350-af66-12c043c777d2 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:13.570800 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba281f0-bc67-490e-9f92-3a08b55a5a57 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:13.571570 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba281f0-bc67-490e-9f92-3a08b55a5a57 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:13.572035 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba281f0-bc67-490e-9f92-3a08b55a5a57 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:13.573676 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 442/885] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:25:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:13.581052 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c2f0f-5799-4850-8279-3f7022b1d0ea None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:13.581330 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c2f0f-5799-4850-8279-3f7022b1d0ea None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:13.581601 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c2f0f-5799-4850-8279-3f7022b1d0ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:13.607436 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-546c2f0f-5799-4850-8279-3f7022b1d0ea None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:13.609481 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-546c2f0f-5799-4850-8279-3f7022b1d0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:13.624285 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:13.651894 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:13.656035 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f562a5-527b-4350-af66-12c043c777d2 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:13.703514 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 443/886] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:13 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:13.708039 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbbf28a-cba8-4856-b934-ba80111c7ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:13.708715 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbbf28a-cba8-4856-b934-ba80111c7ff0 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:13.709063 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbbf28a-cba8-4856-b934-ba80111c7ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:13.709395 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbbf28a-cba8-4856-b934-ba80111c7ff0 None admin] PATH_INFO: `/v3/services` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:13.710283 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbbf28a-cba8-4856-b934-ba80111c7ff0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:13.710589 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbbf28a-cba8-4856-b934-ba80111c7ff0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:13.712893 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbbf28a-cba8-4856-b934-ba80111c7ff0 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:13.718252 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 444/887] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 18 16:25:13 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:13.723896 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546c2f0f-5799-4850-8279-3f7022b1d0ea None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:13.735909 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbbf28a-cba8-4856-b934-ba80111c7ff0 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:13.858416 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546c2f0f-5799-4850-8279-3f7022b1d0ea None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:13.867315 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbbf28a-cba8-4856-b934-ba80111c7ff0 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:13.903605 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f3b5bb-5382-4786-ba77-7c57cec96d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:13.904670 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3b5bb-5382-4786-ba77-7c57cec96d5c None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:13.904854 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36e0932-140a-457e-a845-b3df182129dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:13.905258 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3b5bb-5382-4786-ba77-7c57cec96d5c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:13.905792 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3b5bb-5382-4786-ba77-7c57cec96d5c None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:13.905902 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36e0932-140a-457e-a845-b3df182129dd None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:13.906752 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f3b5bb-5382-4786-ba77-7c57cec96d5c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:13.907173 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36e0932-140a-457e-a845-b3df182129dd None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:13.907429 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f3b5bb-5382-4786-ba77-7c57cec96d5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:13.907605 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36e0932-140a-457e-a845-b3df182129dd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:13.909353 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f3b5bb-5382-4786-ba77-7c57cec96d5c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:13.910831 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:13.921994 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36e0932-140a-457e-a845-b3df182129dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:25:13.922295 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 444/888] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 18 16:25:13 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:13.932726 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f3b5bb-5382-4786-ba77-7c57cec96d5c None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:13.932873 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-d36e0932-140a-457e-a845-b3df182129dd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 16:25:13.932873 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:13.932873 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:13.932873 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:13.932873 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:13.932873 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:13.932873 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:13.932873 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:13.932873 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:13.932873 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:13.932873 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:13.932873 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:13.932873 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:25:13.932873 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:25:13.932873 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:25:13.932873 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:25:13.933788 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:25:13.933788 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:25:13.933788 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:25:13.933788 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:25:13.933788 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:13.933788 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:13.933788 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:13.933788 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:13.933788 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:13.933788 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:13.933788 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:13.933788 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:13.933788 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:13.933788 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:13.933788 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:13.933788 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:13.934631 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:13.934631 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:13.934631 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:13.934631 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:13.934631 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:25:13.934631 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:25:13.934631 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:25:13.934631 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:25:13.934631 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:25:13.934631 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:25:13.934631 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 16:25:13.934631 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:13.934631 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 445/889] 10.222.0.26 () {56 vars in 1120 bytes} [Mon Jul 18 16:25:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:13.939163 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36e0932-140a-457e-a845-b3df182129dd None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:13.941747 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:14.043955 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f3b5bb-5382-4786-ba77-7c57cec96d5c None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:14.053264 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36e0932-140a-457e-a845-b3df182129dd None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:14.080855 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f089a7-8adf-4a92-a072-88c777396362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:14.081812 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f089a7-8adf-4a92-a072-88c777396362 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:14.082269 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f089a7-8adf-4a92-a072-88c777396362 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:14.082714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f089a7-8adf-4a92-a072-88c777396362 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:14.083713 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f089a7-8adf-4a92-a072-88c777396362 None admin] RBAC: Adding query filter params (interface=public, service_id=f138cc7d56694228b1942873359a8b55, region_id=RegionOne) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:14.084228 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f089a7-8adf-4a92-a072-88c777396362 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:14.085781 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f089a7-8adf-4a92-a072-88c777396362 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:14.091200 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889e1556-f3ac-4a96-b247-87ad8aee33f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:14.092378 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e1556-f3ac-4a96-b247-87ad8aee33f4 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:14.092887 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e1556-f3ac-4a96-b247-87ad8aee33f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:14.093324 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e1556-f3ac-4a96-b247-87ad8aee33f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:14.094844 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889e1556-f3ac-4a96-b247-87ad8aee33f4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:14.095387 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889e1556-f3ac-4a96-b247-87ad8aee33f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:14.096932 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889e1556-f3ac-4a96-b247-87ad8aee33f4 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:14.101666 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 445/890] 10.222.0.26 () {56 vars in 1251 bytes} [Mon Jul 18 16:25:13 2022] GET /identity/v3/endpoints?service_id=f138cc7d56694228b1942873359a8b55&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:14.103978 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f089a7-8adf-4a92-a072-88c777396362 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:14.113236 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 446/891] 10.222.0.26 () {56 vars in 1115 bytes} [Mon Jul 18 16:25:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:14.114427 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889e1556-f3ac-4a96-b247-87ad8aee33f4 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:14.177919 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:14.181419 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:14.209769 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f089a7-8adf-4a92-a072-88c777396362 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:14.226174 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889e1556-f3ac-4a96-b247-87ad8aee33f4 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:14.248304 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e409c6-36f3-4531-800a-e2e3e02cc972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:14.248935 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e409c6-36f3-4531-800a-e2e3e02cc972 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:14.249312 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e409c6-36f3-4531-800a-e2e3e02cc972 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:14.249698 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e409c6-36f3-4531-800a-e2e3e02cc972 None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:14.250475 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e409c6-36f3-4531-800a-e2e3e02cc972 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:14.251961 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e409c6-36f3-4531-800a-e2e3e02cc972 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:14.263789 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 446/892] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Jul 18 16:25:14 2022] GET /identity/v3/services => generated 1268 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:25:14.264112 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91b222f-8f9a-4396-9692-155d36c158c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:14.265130 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b222f-8f9a-4396-9692-155d36c158c1 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:14.267008 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b222f-8f9a-4396-9692-155d36c158c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:14.267539 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b222f-8f9a-4396-9692-155d36c158c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:14.283453 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91b222f-8f9a-4396-9692-155d36c158c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:14.285434 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91b222f-8f9a-4396-9692-155d36c158c1 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:14.307575 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-e91b222f-8f9a-4396-9692-155d36c158c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:14.307575 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:14.307575 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:14.307575 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:14.307575 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:14.307575 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:14.307575 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:14.307575 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:14.307575 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:14.307575 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:14.307575 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:14.307575 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:14.307575 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:25:14.307575 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:25:14.307575 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:25:14.309017 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:25:14.309017 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:14.309017 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:14.309017 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:14.309017 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:14.309017 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:14.309017 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:14.309017 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:14.309017 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:14.309017 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:14.309017 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:14.309017 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:14.309017 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:14.309017 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:14.309017 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:14.309017 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:14.309017 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:14.310462 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:25:14.310462 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:25:14.310462 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:14.310462 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:14.312571 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 447/893] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:14.318720 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e409c6-36f3-4531-800a-e2e3e02cc972 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:14.452285 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e409c6-36f3-4531-800a-e2e3e02cc972 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:14.490712 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f64a40-1667-4965-bb1c-275630ad05a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:14.491647 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f64a40-1667-4965-bb1c-275630ad05a3 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:14.492077 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f64a40-1667-4965-bb1c-275630ad05a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:14.492539 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f64a40-1667-4965-bb1c-275630ad05a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:14.493481 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f64a40-1667-4965-bb1c-275630ad05a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:14.493993 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f64a40-1667-4965-bb1c-275630ad05a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:14.495567 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f64a40-1667-4965-bb1c-275630ad05a3 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:14.498457 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:14.513049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 447/894] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:25:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:14.521963 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91b222f-8f9a-4396-9692-155d36c158c1 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:14.537120 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe6b81-174e-4878-8092-00d80a1a2d57 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:14.537600 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe6b81-174e-4878-8092-00d80a1a2d57 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:14.538908 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe6b81-174e-4878-8092-00d80a1a2d57 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:14.541809 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 448/895] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:25:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:14.547938 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9d252-82f6-4d82-90e9-3dc5991b6452 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:14.548220 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9d252-82f6-4d82-90e9-3dc5991b6452 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:14.548500 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9d252-82f6-4d82-90e9-3dc5991b6452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:14.568136 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-d5e9d252-82f6-4d82-90e9-3dc5991b6452 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:14.569245 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d5e9d252-82f6-4d82-90e9-3dc5991b6452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:14.655763 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 449/896] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:14 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:14.662495 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91b222f-8f9a-4396-9692-155d36c158c1 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:14.673778 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:14.675857 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e9d252-82f6-4d82-90e9-3dc5991b6452 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:14.706896 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79567e61-d1ea-426d-8c55-846d448d65ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:14.707986 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79567e61-d1ea-426d-8c55-846d448d65ee None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:14.708264 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79567e61-d1ea-426d-8c55-846d448d65ee None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:14.708538 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79567e61-d1ea-426d-8c55-846d448d65ee None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:14.719844 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79567e61-d1ea-426d-8c55-846d448d65ee None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:14.722158 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79567e61-d1ea-426d-8c55-846d448d65ee None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:14.745404 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-79567e61-d1ea-426d-8c55-846d448d65ee None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 16:25:14.745404 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:14.745404 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:14.745404 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:14.745404 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:14.745404 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:14.745404 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:14.745404 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:14.745404 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:14.745404 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:14.745404 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:14.745404 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:14.745404 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:25:14.745404 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:25:14.745404 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:25:14.745404 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:25:14.746478 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:14.746478 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:14.746478 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:25:14.746478 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:14.746478 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:25:14.746478 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:14.746478 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:14.746478 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:14.746478 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:14.746478 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:14.746478 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:14.746478 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:14.746478 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:14.746478 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:14.746478 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:14.746478 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:14.746478 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:14.746478 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:14.747430 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:14.747430 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:14.747430 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:25:14.747430 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:25:14.747430 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:25:14.747430 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:25:14.747430 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:25:14.747430 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:25:14.747430 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 16:25:14.747430 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:14.750434 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 448/897] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Jul 18 16:25:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:14.756775 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79567e61-d1ea-426d-8c55-846d448d65ee None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:14.810816 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e9d252-82f6-4d82-90e9-3dc5991b6452 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:14.846251 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:14.855745 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c067ef-5e9a-46bd-a0f8-ac8675165626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:14.860669 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c067ef-5e9a-46bd-a0f8-ac8675165626 None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:14.860669 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c067ef-5e9a-46bd-a0f8-ac8675165626 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:14.860669 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c067ef-5e9a-46bd-a0f8-ac8675165626 None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:14.860669 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c067ef-5e9a-46bd-a0f8-ac8675165626 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:14.860669 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c067ef-5e9a-46bd-a0f8-ac8675165626 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:14.898254 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79567e61-d1ea-426d-8c55-846d448d65ee None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:14.937372 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a21826-c6e5-458e-9a3f-14b0f24a22c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:14.938063 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a21826-c6e5-458e-9a3f-14b0f24a22c6 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:14.938343 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a21826-c6e5-458e-9a3f-14b0f24a22c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:14.938620 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a21826-c6e5-458e-9a3f-14b0f24a22c6 None admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:14.939462 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a21826-c6e5-458e-9a3f-14b0f24a22c6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:14.939822 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a21826-c6e5-458e-9a3f-14b0f24a22c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:14.941407 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a21826-c6e5-458e-9a3f-14b0f24a22c6 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:14.962701 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 449/898] 10.222.0.26 () {56 vars in 1149 bytes} [Mon Jul 18 16:25:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:14.964261 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8396d7fb-ca31-450a-a446-bd0b72cbc49f None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:14.964586 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8396d7fb-ca31-450a-a446-bd0b72cbc49f None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:14.964879 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8396d7fb-ca31-450a-a446-bd0b72cbc49f None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:14.966398 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 450/899] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:25:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:14.971624 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8396d7fb-ca31-450a-a446-bd0b72cbc49f None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:15.004544 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:15.103468 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8396d7fb-ca31-450a-a446-bd0b72cbc49f None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:15.129392 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecc049b-fc02-40ad-9640-5bd9066aa49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:15.130039 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecc049b-fc02-40ad-9640-5bd9066aa49d None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:15.130318 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecc049b-fc02-40ad-9640-5bd9066aa49d None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:15.130589 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecc049b-fc02-40ad-9640-5bd9066aa49d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:15.171875 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 450/900] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jul 18 16:25:14 2022] POST /identity/v3/services => generated 241 bytes in 496 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:25:15.185298 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e0026-05c8-442d-8f31-aae5272b7e7d None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:15.185587 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e0026-05c8-442d-8f31-aae5272b7e7d None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:15.185898 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e0026-05c8-442d-8f31-aae5272b7e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:15.211696 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecc049b-fc02-40ad-9640-5bd9066aa49d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:15.213218 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecc049b-fc02-40ad-9640-5bd9066aa49d None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:15.222325 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-811e0026-05c8-442d-8f31-aae5272b7e7d None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:15.224354 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-811e0026-05c8-442d-8f31-aae5272b7e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:15.266155 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:15.334791 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 451/901] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:15.358507 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811e0026-05c8-442d-8f31-aae5272b7e7d None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:15.379057 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 451/902] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:25:14 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 16:25:15.386417 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecc049b-fc02-40ad-9640-5bd9066aa49d None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:15.495422 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811e0026-05c8-442d-8f31-aae5272b7e7d None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:15.510874 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:15.537778 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d7c9ee-11b5-4776-bc2d-9535ab7d3e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:15.538196 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecc049b-fc02-40ad-9640-5bd9066aa49d None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:15.538770 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7c9ee-11b5-4776-bc2d-9535ab7d3e0c None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:15.539122 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7c9ee-11b5-4776-bc2d-9535ab7d3e0c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:15.540104 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7c9ee-11b5-4776-bc2d-9535ab7d3e0c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:15.541046 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d7c9ee-11b5-4776-bc2d-9535ab7d3e0c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:15.543809 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d7c9ee-11b5-4776-bc2d-9535ab7d3e0c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:15.559859 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-62d7c9ee-11b5-4776-bc2d-9535ab7d3e0c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 16:25:15.559859 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:15.559859 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:25:15.559859 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:25:15.559859 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:25:15.559859 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:25:15.559859 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:25:15.559859 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:15.559859 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 16:25:15.559859 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:15.559859 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:25:15.559859 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:15.559859 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:15.559859 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:15.559859 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:15.559859 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:15.560847 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:15.560847 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:15.560847 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:15.560847 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:15.560847 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:15.560847 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:15.560847 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:15.560847 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:25:15.560847 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:25:15.560847 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:25:15.560847 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:25:15.560847 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:15.560847 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:15.560847 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:15.560847 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:15.560847 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:15.561678 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:15.561678 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:15.561678 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:15.561678 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:15.561678 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:15.561678 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:15.561678 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:15.561678 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:15.561678 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:15.561678 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:15.561678 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:15.561678 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:25:15.561678 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:15.561678 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 16:25:15.561678 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:15.562443 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 452/903] 10.222.0.26 () {56 vars in 1132 bytes} [Mon Jul 18 16:25:15 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:15.568872 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d7c9ee-11b5-4776-bc2d-9535ab7d3e0c None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:15.573044 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:15.582368 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d460a4-234c-49a7-a012-717da8811021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:15.583263 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d460a4-234c-49a7-a012-717da8811021 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:15.583648 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d460a4-234c-49a7-a012-717da8811021 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:15.583991 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d460a4-234c-49a7-a012-717da8811021 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:15.600088 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d460a4-234c-49a7-a012-717da8811021 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:15.602649 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d460a4-234c-49a7-a012-717da8811021 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:15.629566 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-22d460a4-234c-49a7-a012-717da8811021 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:15.629566 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:15.629566 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:15.629566 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:15.629566 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:15.629566 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:15.629566 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:15.629566 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:15.629566 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:15.629566 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:15.629566 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:15.629566 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:15.629566 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:25:15.629566 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:25:15.629566 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:25:15.630662 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:25:15.630662 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:15.630662 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:15.630662 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:15.630662 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:15.630662 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:15.630662 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:15.630662 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:15.630662 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:15.630662 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:15.630662 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:15.630662 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:15.630662 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:15.630662 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:15.630662 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:15.630662 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:15.630662 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:15.631768 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:25:15.631768 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:25:15.631768 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:15.631768 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:15.632726 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 452/904] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:25:15.639287 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d460a4-234c-49a7-a012-717da8811021 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:15.714452 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d7c9ee-11b5-4776-bc2d-9535ab7d3e0c None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:15.764538 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7f9e8c-c04e-451f-b28c-35b47e6c7f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:15.765378 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f9e8c-c04e-451f-b28c-35b47e6c7f68 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:15.765758 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f9e8c-c04e-451f-b28c-35b47e6c7f68 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:15.766164 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f9e8c-c04e-451f-b28c-35b47e6c7f68 None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:15.766978 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7f9e8c-c04e-451f-b28c-35b47e6c7f68 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:15.767377 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7f9e8c-c04e-451f-b28c-35b47e6c7f68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:15.769067 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7f9e8c-c04e-451f-b28c-35b47e6c7f68 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:15.779590 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 453/905] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Jul 18 16:25:15 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:15.782035 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7f9e8c-c04e-451f-b28c-35b47e6c7f68 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:15.792686 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d460a4-234c-49a7-a012-717da8811021 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:15.834614 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841b1e47-e728-4a1c-b761-4a3649260649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:15.835415 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b1e47-e728-4a1c-b761-4a3649260649 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:15.835813 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b1e47-e728-4a1c-b761-4a3649260649 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:15.836691 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b1e47-e728-4a1c-b761-4a3649260649 None admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:15.838277 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841b1e47-e728-4a1c-b761-4a3649260649 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:15.838695 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841b1e47-e728-4a1c-b761-4a3649260649 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:15.842148 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841b1e47-e728-4a1c-b761-4a3649260649 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:15.843244 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:15.863243 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 453/906] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:25:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:15.864699 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841b1e47-e728-4a1c-b761-4a3649260649 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:15.925524 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:15.926143 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7f9e8c-c04e-451f-b28c-35b47e6c7f68 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:15.964941 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ca1517-4d14-4d78-b78b-614bdc394020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:15.965742 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca1517-4d14-4d78-b78b-614bdc394020 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:15.966131 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca1517-4d14-4d78-b78b-614bdc394020 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:15.966475 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca1517-4d14-4d78-b78b-614bdc394020 None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:15.967274 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ca1517-4d14-4d78-b78b-614bdc394020 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:15.967773 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ca1517-4d14-4d78-b78b-614bdc394020 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:15.969265 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ca1517-4d14-4d78-b78b-614bdc394020 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:15.975530 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 454/907] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Jul 18 16:25:15 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:15.983383 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ca1517-4d14-4d78-b78b-614bdc394020 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:16.009334 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841b1e47-e728-4a1c-b761-4a3649260649 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:16.062807 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe6db35-22af-472b-954c-ccbd5b535e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:16.064379 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe6db35-22af-472b-954c-ccbd5b535e6b None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:16.064955 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe6db35-22af-472b-954c-ccbd5b535e6b None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:16.065505 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe6db35-22af-472b-954c-ccbd5b535e6b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:16.075744 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe6db35-22af-472b-954c-ccbd5b535e6b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:16.078576 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe6db35-22af-472b-954c-ccbd5b535e6b None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:16.097836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-ffe6db35-22af-472b-954c-ccbd5b535e6b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:25:16.097836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:16.097836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:16.097836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:16.097836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:16.097836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:16.097836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:16.097836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:16.097836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:16.097836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:16.097836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:16.097836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:16.097836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:25:16.097836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:25:16.097836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:25:16.098988 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:25:16.098988 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:16.098988 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:16.098988 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:16.098988 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:16.098988 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:16.098988 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:16.098988 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:16.098988 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:16.098988 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:16.098988 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:16.098988 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:16.098988 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:16.098988 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:16.098988 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:16.098988 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:16.098988 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:16.100076 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:25:16.100076 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:25:16.100076 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:25:16.100076 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:25:16.100076 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:25:16.100076 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:25:16.100076 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:25:16.100076 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:16.107424 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 454/908] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:25:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:16.109057 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe6db35-22af-472b-954c-ccbd5b535e6b None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:16.123949 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ca1517-4d14-4d78-b78b-614bdc394020 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:16.127134 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:16.172286 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cffce21-06a6-4c1a-8641-9370b4ad71d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:16.173295 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cffce21-06a6-4c1a-8641-9370b4ad71d3 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:16.173588 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cffce21-06a6-4c1a-8641-9370b4ad71d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:16.173867 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cffce21-06a6-4c1a-8641-9370b4ad71d3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:16.174665 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cffce21-06a6-4c1a-8641-9370b4ad71d3 None admin] RBAC: Adding query filter params (interface=public, service_id=b5b71865705e4e77bb696088f34cd855, region_id=RegionOne) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:16.174977 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cffce21-06a6-4c1a-8641-9370b4ad71d3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:16.176626 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cffce21-06a6-4c1a-8641-9370b4ad71d3 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:16.187565 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 455/909] 10.222.0.26 () {56 vars in 1251 bytes} [Mon Jul 18 16:25:15 2022] GET /identity/v3/endpoints?service_id=b5b71865705e4e77bb696088f34cd855&interface=public®ion_id=RegionOne => generated 194 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:16.189892 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cffce21-06a6-4c1a-8641-9370b4ad71d3 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:16.245268 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:16.249287 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe6db35-22af-472b-954c-ccbd5b535e6b None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:16.288214 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e356222a-9fd4-41db-b276-56cb0d7fa65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:16.289342 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e356222a-9fd4-41db-b276-56cb0d7fa65c None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:16.290430 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e356222a-9fd4-41db-b276-56cb0d7fa65c None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:16.290922 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e356222a-9fd4-41db-b276-56cb0d7fa65c None admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:16.291983 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e356222a-9fd4-41db-b276-56cb0d7fa65c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:16.292506 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e356222a-9fd4-41db-b276-56cb0d7fa65c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:16.294510 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e356222a-9fd4-41db-b276-56cb0d7fa65c None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:16.315976 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 455/910] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 18 16:25:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:16.317656 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e356222a-9fd4-41db-b276-56cb0d7fa65c None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:16.323902 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cffce21-06a6-4c1a-8641-9370b4ad71d3 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:16.370264 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566d2586-63de-4c1b-ba30-006ac6f0c4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:16.370961 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d2586-63de-4c1b-ba30-006ac6f0c4c5 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:16.371249 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d2586-63de-4c1b-ba30-006ac6f0c4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:16.371550 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d2586-63de-4c1b-ba30-006ac6f0c4c5 None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:16.372248 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566d2586-63de-4c1b-ba30-006ac6f0c4c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:16.373897 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566d2586-63de-4c1b-ba30-006ac6f0c4c5 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:16.383617 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 456/911] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Jul 18 16:25:16 2022] GET /identity/v3/services => generated 1497 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:25:16.397964 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e4898-2efe-490a-bf8e-0472a83eb854 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:16.398322 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e4898-2efe-490a-bf8e-0472a83eb854 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:16.398716 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e4898-2efe-490a-bf8e-0472a83eb854 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:16.400638 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 457/912] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:25:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:25:16.406820 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af7f345-ba40-41bf-abbf-16feb9b4b69c None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:16.407235 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af7f345-ba40-41bf-abbf-16feb9b4b69c None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:16.407391 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af7f345-ba40-41bf-abbf-16feb9b4b69c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:16.427698 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-3af7f345-ba40-41bf-abbf-16feb9b4b69c None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:16.428861 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-3af7f345-ba40-41bf-abbf-16feb9b4b69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:16.449529 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e356222a-9fd4-41db-b276-56cb0d7fa65c None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:16.462252 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:16.493185 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e6a5c-e292-4ee8-ba33-ad20d9157886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:16.494368 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e6a5c-e292-4ee8-ba33-ad20d9157886 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:16.494822 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e6a5c-e292-4ee8-ba33-ad20d9157886 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:16.496208 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e6a5c-e292-4ee8-ba33-ad20d9157886 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:16.497359 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072e6a5c-e292-4ee8-ba33-ad20d9157886 None admin] RBAC: Adding query filter params (role.id=3394397b06be42fe9507041df2452f79, scope.project.id=eb9747e2299c4e0991fd7edb4462c7e5, user.id=da799b0044f54da6835aa06e43ce1a6f) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:16.497860 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072e6a5c-e292-4ee8-ba33-ad20d9157886 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:16.499739 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072e6a5c-e292-4ee8-ba33-ad20d9157886 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:16.507197 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 456/913] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 18 16:25:16 2022] GET /identity/v3/role_assignments?user.id=da799b0044f54da6835aa06e43ce1a6f&scope.project.id=eb9747e2299c4e0991fd7edb4462c7e5&role.id=3394397b06be42fe9507041df2452f79 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:16.519722 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 458/914] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:16.539102 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e6a5c-e292-4ee8-ba33-ad20d9157886 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:16.572497 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:16.666885 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e6a5c-e292-4ee8-ba33-ad20d9157886 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:16.708616 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d82409-66fc-4bd9-9d68-da375785823d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:16.709469 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d82409-66fc-4bd9-9d68-da375785823d None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:16.709816 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d82409-66fc-4bd9-9d68-da375785823d None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:16.710091 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d82409-66fc-4bd9-9d68-da375785823d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:16.710776 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d82409-66fc-4bd9-9d68-da375785823d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:16.712911 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d82409-66fc-4bd9-9d68-da375785823d None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:16.725786 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-38d82409-66fc-4bd9-9d68-da375785823d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 16:25:16.725786 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:16.725786 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:25:16.725786 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:25:16.725786 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:25:16.725786 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:25:16.725786 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:25:16.725786 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:16.725786 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 16:25:16.725786 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:16.725786 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:25:16.725786 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:16.725786 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:16.725786 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:16.725786 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:16.725786 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:16.725786 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:16.727684 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:16.727684 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:16.727684 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:16.727684 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:16.727684 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:16.727684 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:16.727684 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:25:16.727684 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:25:16.727684 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:25:16.727684 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:25:16.727684 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:16.727684 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:16.727684 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:16.727684 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:16.727684 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:16.727684 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:16.728919 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:16.728919 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:16.728919 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:16.728919 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:16.728919 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:16.728919 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:16.728919 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:16.728919 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:16.728919 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:16.728919 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:16.728919 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:25:16.728919 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:16.728919 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 16:25:16.728919 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:16.728919 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 457/915] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 18 16:25:16 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:16.734107 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af7f345-ba40-41bf-abbf-16feb9b4b69c None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:16.850310 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af7f345-ba40-41bf-abbf-16feb9b4b69c None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:16.890735 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764749bd-ba4a-4133-a4e8-cba848828123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:16.891426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764749bd-ba4a-4133-a4e8-cba848828123 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:16.891743 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764749bd-ba4a-4133-a4e8-cba848828123 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:16.892026 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764749bd-ba4a-4133-a4e8-cba848828123 None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:16.892752 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764749bd-ba4a-4133-a4e8-cba848828123 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:16.893058 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764749bd-ba4a-4133-a4e8-cba848828123 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:16.894470 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764749bd-ba4a-4133-a4e8-cba848828123 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:16.905002 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 459/916] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 18 16:25:16 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:16.906971 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d82409-66fc-4bd9-9d68-da375785823d None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:17.014760 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:17.028112 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d82409-66fc-4bd9-9d68-da375785823d None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:17.069189 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5cce0b-ae7d-4a0d-b253-823a3a87b596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:17.069878 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5cce0b-ae7d-4a0d-b253-823a3a87b596 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:17.070162 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5cce0b-ae7d-4a0d-b253-823a3a87b596 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:17.070508 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5cce0b-ae7d-4a0d-b253-823a3a87b596 None admin] PATH_INFO: `/v3/services` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:17.071223 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5cce0b-ae7d-4a0d-b253-823a3a87b596 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:17.071669 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5cce0b-ae7d-4a0d-b253-823a3a87b596 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:17.073061 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5cce0b-ae7d-4a0d-b253-823a3a87b596 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:17.083477 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 458/917] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 18 16:25:16 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:17.084884 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764749bd-ba4a-4133-a4e8-cba848828123 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:17.116109 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:17.190956 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764749bd-ba4a-4133-a4e8-cba848828123 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:17.226875 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c468349-b08a-4d93-abb9-889190aa9c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:17.228130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c468349-b08a-4d93-abb9-889190aa9c1c None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:17.228418 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c468349-b08a-4d93-abb9-889190aa9c1c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:17.228695 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c468349-b08a-4d93-abb9-889190aa9c1c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:17.229453 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c468349-b08a-4d93-abb9-889190aa9c1c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:17.230856 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c468349-b08a-4d93-abb9-889190aa9c1c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:17.319715 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 460/918] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Jul 18 16:25:17 2022] POST /identity/v3/endpoints => generated 346 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:25:17.425146 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3999ffcf-dc94-47fe-947d-57303b64de7a None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:17.425506 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3999ffcf-dc94-47fe-947d-57303b64de7a None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:17.425712 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3999ffcf-dc94-47fe-947d-57303b64de7a None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:17.427412 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 459/919] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:25:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:17.433378 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8fce9f-3f6e-4706-900c-e09b7dbe9bd1 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:17.433927 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8fce9f-3f6e-4706-900c-e09b7dbe9bd1 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:17.433927 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8fce9f-3f6e-4706-900c-e09b7dbe9bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:17.452456 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-7d8fce9f-3f6e-4706-900c-e09b7dbe9bd1 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:17.453677 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-7d8fce9f-3f6e-4706-900c-e09b7dbe9bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:17.532747 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 461/920] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:17 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:17.555506 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3999ffcf-dc94-47fe-947d-57303b64de7a None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:17.675368 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3999ffcf-dc94-47fe-947d-57303b64de7a None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:17.716802 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0936f12b-3f89-4b70-9b24-9f8097cdccd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:17.718036 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0936f12b-3f89-4b70-9b24-9f8097cdccd9 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:17.718523 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0936f12b-3f89-4b70-9b24-9f8097cdccd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:17.718993 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0936f12b-3f89-4b70-9b24-9f8097cdccd9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:17.719889 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0936f12b-3f89-4b70-9b24-9f8097cdccd9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:17.721502 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0936f12b-3f89-4b70-9b24-9f8097cdccd9 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:17.736310 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-0936f12b-3f89-4b70-9b24-9f8097cdccd9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 16:25:17.736310 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:17.736310 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:25:17.736310 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:25:17.736310 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:25:17.736310 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:25:17.736310 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:25:17.736310 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:17.736310 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 16:25:17.736310 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:17.736310 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:25:17.736310 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:17.736310 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:17.736310 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:17.736310 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:17.736310 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:17.736310 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:17.737355 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:17.737355 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:17.737355 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:17.737355 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:17.737355 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:17.737355 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:17.737355 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:25:17.737355 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:25:17.737355 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:25:17.737355 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:25:17.737355 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:17.737355 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:17.737355 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:17.737355 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:17.737355 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:17.737355 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:17.738177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:17.738177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:17.738177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:17.738177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:17.738177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:17.738177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:17.738177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:17.738177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:17.738177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:17.738177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:17.738177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:25:17.738177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:17.738177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 16:25:17.738177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:17.741379 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 460/921] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Jul 18 16:25:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:17.747961 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8fce9f-3f6e-4706-900c-e09b7dbe9bd1 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:17.759555 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:17.864471 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8fce9f-3f6e-4706-900c-e09b7dbe9bd1 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:17.905886 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee355b-566f-405e-8d91-63178c110da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:17.906556 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee355b-566f-405e-8d91-63178c110da6 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:17.906844 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee355b-566f-405e-8d91-63178c110da6 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:17.907124 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee355b-566f-405e-8d91-63178c110da6 None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:17.907867 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee355b-566f-405e-8d91-63178c110da6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:17.908172 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee355b-566f-405e-8d91-63178c110da6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:17.909534 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee355b-566f-405e-8d91-63178c110da6 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:17.915974 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 462/922] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Jul 18 16:25:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:17.922006 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0936f12b-3f89-4b70-9b24-9f8097cdccd9 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:17.953852 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:18.044004 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0936f12b-3f89-4b70-9b24-9f8097cdccd9 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:18.081229 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e919a3-1f5e-40d3-8d4a-fe35eba7e12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:18.082315 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e919a3-1f5e-40d3-8d4a-fe35eba7e12a None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:18.082847 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e919a3-1f5e-40d3-8d4a-fe35eba7e12a None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:18.083318 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e919a3-1f5e-40d3-8d4a-fe35eba7e12a None admin] PATH_INFO: `/v3/services` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:18.084286 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e919a3-1f5e-40d3-8d4a-fe35eba7e12a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:18.084788 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e919a3-1f5e-40d3-8d4a-fe35eba7e12a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:18.086662 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e919a3-1f5e-40d3-8d4a-fe35eba7e12a None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:18.092655 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 461/923] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Jul 18 16:25:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:18.100873 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee355b-566f-405e-8d91-63178c110da6 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:18.176522 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:18.207897 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee355b-566f-405e-8d91-63178c110da6 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:18.245757 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebc5637-e2f9-4177-80b4-8aff13a6cb21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:18.246424 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc5637-e2f9-4177-80b4-8aff13a6cb21 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:18.246708 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc5637-e2f9-4177-80b4-8aff13a6cb21 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:18.246990 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc5637-e2f9-4177-80b4-8aff13a6cb21 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:18.247810 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebc5637-e2f9-4177-80b4-8aff13a6cb21 None admin] RBAC: Adding query filter params (interface=public, service_id=53fcca302b884a29b289944126fa2d24, region_id=RegionOne) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:18.248144 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebc5637-e2f9-4177-80b4-8aff13a6cb21 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:18.249527 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebc5637-e2f9-4177-80b4-8aff13a6cb21 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:18.255580 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 463/924] 10.222.0.26 () {56 vars in 1251 bytes} [Mon Jul 18 16:25:18 2022] GET /identity/v3/endpoints?service_id=53fcca302b884a29b289944126fa2d24&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:18.261522 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e919a3-1f5e-40d3-8d4a-fe35eba7e12a None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:18.362319 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:18.383382 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e919a3-1f5e-40d3-8d4a-fe35eba7e12a None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:18.419534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed6e890-5c51-4b97-a95c-32b1ede3e26f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:18.420247 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6e890-5c51-4b97-a95c-32b1ede3e26f None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:18.420566 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6e890-5c51-4b97-a95c-32b1ede3e26f None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:18.420864 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6e890-5c51-4b97-a95c-32b1ede3e26f None admin] PATH_INFO: `/v3/services` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:18.421631 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed6e890-5c51-4b97-a95c-32b1ede3e26f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:18.423056 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed6e890-5c51-4b97-a95c-32b1ede3e26f None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:18.430443 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 462/925] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Jul 18 16:25:18 2022] GET /identity/v3/services => generated 1497 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:25:18.699845 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd589b37-0ce0-4ebc-898c-353f5bea114b None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:18.700376 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd589b37-0ce0-4ebc-898c-353f5bea114b None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:18.700376 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd589b37-0ce0-4ebc-898c-353f5bea114b None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:18.702150 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 464/926] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:25:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:18.707474 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198f9918-924e-4af7-861c-44b5badbc79e None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:18.707885 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198f9918-924e-4af7-861c-44b5badbc79e None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:18.708238 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198f9918-924e-4af7-861c-44b5badbc79e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:18.726150 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-198f9918-924e-4af7-861c-44b5badbc79e None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:18.727538 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-198f9918-924e-4af7-861c-44b5badbc79e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:18.780278 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b518c7-07d9-4b1f-b4c7-2b4a99ca7240 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:18.780608 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b518c7-07d9-4b1f-b4c7-2b4a99ca7240 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:18.780863 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b518c7-07d9-4b1f-b4c7-2b4a99ca7240 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:18.782494 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 465/927] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:25:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:25:18.787536 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1966e3fb-1e12-488a-aa70-5740abd52cc1 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:18.787826 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1966e3fb-1e12-488a-aa70-5740abd52cc1 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:18.788099 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1966e3fb-1e12-488a-aa70-5740abd52cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:18.800971 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 463/928] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:18 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:18.806503 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-1966e3fb-1e12-488a-aa70-5740abd52cc1 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:18.807717 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-1966e3fb-1e12-488a-aa70-5740abd52cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:18.824677 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198f9918-924e-4af7-861c-44b5badbc79e None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:18.915732 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 466/929] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:18 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:25:18.921957 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:18.938796 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1966e3fb-1e12-488a-aa70-5740abd52cc1 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:18.952947 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198f9918-924e-4af7-861c-44b5badbc79e None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:19.005022 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c7acd9-0bbc-4531-857d-fcdecdf2b174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:19.006787 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c7acd9-0bbc-4531-857d-fcdecdf2b174 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:19.007262 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c7acd9-0bbc-4531-857d-fcdecdf2b174 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:19.008713 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c7acd9-0bbc-4531-857d-fcdecdf2b174 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:19.010070 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c7acd9-0bbc-4531-857d-fcdecdf2b174 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:19.012288 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c7acd9-0bbc-4531-857d-fcdecdf2b174 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:19.032467 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-95c7acd9-0bbc-4531-857d-fcdecdf2b174 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 16:25:19.032467 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:19.032467 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:25:19.032467 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:25:19.032467 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:25:19.032467 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:25:19.032467 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:25:19.032467 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:19.032467 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 16:25:19.032467 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:19.032467 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:25:19.032467 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:19.032467 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:19.032467 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:19.032467 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:19.032467 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:19.033453 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:19.033453 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:19.033453 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:19.033453 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:19.033453 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:19.033453 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:19.033453 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:19.033453 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:25:19.033453 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:25:19.033453 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:25:19.033453 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:25:19.033453 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:19.033453 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:19.033453 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:19.033453 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:19.033453 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:19.034281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:19.034281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:19.034281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:19.034281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:19.034281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:19.034281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:19.034281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:19.034281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:19.034281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:19.034281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:19.034281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:19.034281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:25:19.034281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:19.034281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 16:25:19.034281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:19.035711 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:19.038277 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 464/930] 10.222.0.26 () {56 vars in 1132 bytes} [Mon Jul 18 16:25:18 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:25:19.045146 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c7acd9-0bbc-4531-857d-fcdecdf2b174 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:19.080277 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1966e3fb-1e12-488a-aa70-5740abd52cc1 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:19.117289 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a533b8b-2260-46a5-886a-d25e228400fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:19.118552 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a533b8b-2260-46a5-886a-d25e228400fe None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:19.119055 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a533b8b-2260-46a5-886a-d25e228400fe None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:19.119562 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a533b8b-2260-46a5-886a-d25e228400fe None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:19.130711 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a533b8b-2260-46a5-886a-d25e228400fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:25:19.140224 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-1a533b8b-2260-46a5-886a-d25e228400fe None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 16:25:19.140224 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:19.140224 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:19.140224 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:19.140224 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:19.140224 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:19.140224 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:19.140224 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:19.140224 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:19.140224 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:19.140224 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:19.140224 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:19.140224 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:25:19.140224 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:25:19.140224 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:25:19.140224 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:25:19.141177 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:25:19.141177 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:25:19.141177 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:25:19.141177 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:25:19.141177 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:19.141177 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:19.141177 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:19.141177 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:19.141177 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:19.141177 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:19.141177 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:19.141177 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:19.141177 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:19.141177 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:19.141177 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:19.141177 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:19.142143 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:19.142143 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:19.142143 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:19.142143 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:19.142143 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:25:19.142143 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:25:19.142143 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:25:19.142143 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:25:19.142143 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:25:19.142143 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:25:19.142143 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 16:25:19.142143 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:19.144912 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 467/931] 10.222.0.26 () {56 vars in 1120 bytes} [Mon Jul 18 16:25:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:19.151968 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a533b8b-2260-46a5-886a-d25e228400fe None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:19.168113 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c7acd9-0bbc-4531-857d-fcdecdf2b174 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:19.206459 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd914a4-cf0c-4d04-ab19-1607f98141e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:19.207086 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd914a4-cf0c-4d04-ab19-1607f98141e3 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:19.207380 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd914a4-cf0c-4d04-ab19-1607f98141e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:19.207683 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd914a4-cf0c-4d04-ab19-1607f98141e3 None admin] PATH_INFO: `/v3/services` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:19.208399 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd914a4-cf0c-4d04-ab19-1607f98141e3 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:19.208709 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd914a4-cf0c-4d04-ab19-1607f98141e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:19.210204 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd914a4-cf0c-4d04-ab19-1607f98141e3 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:19.222383 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 465/932] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Jul 18 16:25:19 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:19.223625 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd914a4-cf0c-4d04-ab19-1607f98141e3 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:19.227840 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:19.282583 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a533b8b-2260-46a5-886a-d25e228400fe None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:19.311541 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:19.322443 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dce93fa-7291-4679-9f10-9b2597cdcb0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:19.323521 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce93fa-7291-4679-9f10-9b2597cdcb0f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:19.324177 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce93fa-7291-4679-9f10-9b2597cdcb0f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:19.324680 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce93fa-7291-4679-9f10-9b2597cdcb0f None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:19.325705 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dce93fa-7291-4679-9f10-9b2597cdcb0f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:19.326273 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dce93fa-7291-4679-9f10-9b2597cdcb0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:19.328091 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dce93fa-7291-4679-9f10-9b2597cdcb0f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:19.335893 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd914a4-cf0c-4d04-ab19-1607f98141e3 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:19.344270 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 468/933] 10.222.0.26 () {56 vars in 1115 bytes} [Mon Jul 18 16:25:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:19.345796 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dce93fa-7291-4679-9f10-9b2597cdcb0f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:19.383661 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316f9eae-de62-4218-b36a-876b390f2afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:19.384641 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f9eae-de62-4218-b36a-876b390f2afb None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:19.384892 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f9eae-de62-4218-b36a-876b390f2afb None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:19.385159 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f9eae-de62-4218-b36a-876b390f2afb None admin] PATH_INFO: `/v3/services` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:19.385959 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316f9eae-de62-4218-b36a-876b390f2afb None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:19.386266 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316f9eae-de62-4218-b36a-876b390f2afb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:19.387759 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316f9eae-de62-4218-b36a-876b390f2afb None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:19.398001 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 466/934] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Jul 18 16:25:19 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:19.400129 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316f9eae-de62-4218-b36a-876b390f2afb None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:19.485534 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dce93fa-7291-4679-9f10-9b2597cdcb0f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:19.490616 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:19.522432 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c802d1dc-1192-412d-9f92-b680f39b2ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:19.523771 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802d1dc-1192-412d-9f92-b680f39b2ade None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:19.524275 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802d1dc-1192-412d-9f92-b680f39b2ade None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:19.524705 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802d1dc-1192-412d-9f92-b680f39b2ade None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:19.526396 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316f9eae-de62-4218-b36a-876b390f2afb None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:19.536934 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c802d1dc-1192-412d-9f92-b680f39b2ade None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:19.538618 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c802d1dc-1192-412d-9f92-b680f39b2ade None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:19.559279 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-c802d1dc-1192-412d-9f92-b680f39b2ade None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:19.559279 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:19.559279 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:19.559279 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:19.559279 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:19.559279 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:19.559279 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:19.559279 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:19.559279 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:19.559279 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:19.559279 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:19.559279 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:19.559279 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:25:19.559279 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:25:19.559279 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:25:19.560421 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:25:19.560421 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:19.560421 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:19.560421 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:19.560421 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:19.560421 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:19.560421 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:19.560421 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:19.560421 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:19.560421 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:19.560421 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:19.560421 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:19.560421 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:19.560421 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:19.560421 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:19.560421 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:19.560421 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:19.561457 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:25:19.561457 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:25:19.561457 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:19.561457 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:19.561677 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a2ee60-c5e2-4789-8635-e326ae24f268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:19.562140 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 469/935] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:19.562307 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2ee60-c5e2-4789-8635-e326ae24f268 None admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:19.562569 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2ee60-c5e2-4789-8635-e326ae24f268 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:19.562847 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2ee60-c5e2-4789-8635-e326ae24f268 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:19.563437 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609dc64c-2113-41e8-ac45-c383aef33976 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:19.563657 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a2ee60-c5e2-4789-8635-e326ae24f268 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:19.563763 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609dc64c-2113-41e8-ac45-c383aef33976 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:19.564029 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609dc64c-2113-41e8-ac45-c383aef33976 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:19.564991 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a2ee60-c5e2-4789-8635-e326ae24f268 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:19.565494 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 470/936] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:25:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:19.567719 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609dc64c-2113-41e8-ac45-c383aef33976 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:19.614719 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:19.677192 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609dc64c-2113-41e8-ac45-c383aef33976 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:19.695406 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 467/937] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Jul 18 16:25:19 2022] POST /identity/v3/endpoints => generated 347 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:25:19.695406 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1a26d-8eb7-497e-b6b8-cb306ed86af0 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:19.695406 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1a26d-8eb7-497e-b6b8-cb306ed86af0 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:19.695406 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1a26d-8eb7-497e-b6b8-cb306ed86af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:19.731521 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-70f1a26d-8eb7-497e-b6b8-cb306ed86af0 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:19.731521 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-70f1a26d-8eb7-497e-b6b8-cb306ed86af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:19.733882 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03223712-ce29-4c1b-9621-8c85e07729f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:19.735328 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03223712-ce29-4c1b-9621-8c85e07729f3 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:19.735328 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03223712-ce29-4c1b-9621-8c85e07729f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:19.735328 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03223712-ce29-4c1b-9621-8c85e07729f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:19.736024 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03223712-ce29-4c1b-9621-8c85e07729f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:19.736330 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03223712-ce29-4c1b-9621-8c85e07729f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:19.742795 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03223712-ce29-4c1b-9621-8c85e07729f3 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:19.753749 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 471/938] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:25:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:19.759804 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03223712-ce29-4c1b-9621-8c85e07729f3 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:19.840005 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 468/939] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:19.861886 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f1a26d-8eb7-497e-b6b8-cb306ed86af0 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:19.894088 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:19.902989 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03223712-ce29-4c1b-9621-8c85e07729f3 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:19.938807 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:19.956271 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9625f0b8-2b6f-4c26-a59e-3a2a1bea3f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:19.957094 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9625f0b8-2b6f-4c26-a59e-3a2a1bea3f68 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:19.957373 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9625f0b8-2b6f-4c26-a59e-3a2a1bea3f68 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:19.957645 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9625f0b8-2b6f-4c26-a59e-3a2a1bea3f68 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:19.969246 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9625f0b8-2b6f-4c26-a59e-3a2a1bea3f68 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:19.971633 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9625f0b8-2b6f-4c26-a59e-3a2a1bea3f68 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:19.993631 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-9625f0b8-2b6f-4c26-a59e-3a2a1bea3f68 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 16:25:19.993631 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:19.993631 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:19.993631 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:19.993631 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:19.993631 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:19.993631 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:19.993631 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:19.993631 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:19.993631 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:19.993631 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:19.993631 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:19.993631 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:25:19.993631 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:25:19.993631 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:25:19.993631 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:25:19.995322 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:19.995322 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:19.995322 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:25:19.995322 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:19.995322 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:25:19.995322 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:19.995322 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:19.995322 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:19.995322 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:19.995322 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:19.995322 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:19.995322 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:19.995322 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:19.995322 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:19.995322 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:19.995322 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:19.995322 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:19.995322 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:19.997195 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:19.997195 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:19.997195 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:25:19.997195 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:25:19.997195 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:25:19.997195 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:25:19.997195 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:25:19.997195 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:25:19.997195 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 16:25:19.997195 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:19.998281 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 472/940] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Jul 18 16:25:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:20.006141 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9625f0b8-2b6f-4c26-a59e-3a2a1bea3f68 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:20.011737 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f1a26d-8eb7-497e-b6b8-cb306ed86af0 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:20.061461 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524b3fff-c9fd-4c13-a2f4-883d49b74729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:20.062824 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b3fff-c9fd-4c13-a2f4-883d49b74729 None admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:20.063286 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b3fff-c9fd-4c13-a2f4-883d49b74729 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:20.063752 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b3fff-c9fd-4c13-a2f4-883d49b74729 None admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:20.064638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524b3fff-c9fd-4c13-a2f4-883d49b74729 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:20.066248 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524b3fff-c9fd-4c13-a2f4-883d49b74729 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:20.067903 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: WARNING keystone.server.flask.common [None req-524b3fff-c9fd-4c13-a2f4-883d49b74729 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 18 16:25:20.112995 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.sql.core [None req-524b3fff-c9fd-4c13-a2f4-883d49b74729 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 18 16:25:20.112995 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 18 16:25:20.112995 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [parameters: {'id': '511d372c937e4267bcf3b176fb44d9e3', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 18 16:25:20.112995 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80367) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 18 16:25:20.130545 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-524b3fff-c9fd-4c13-a2f4-883d49b74729 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 511d372c937e4267bcf3b176fb44d9e3.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 511d372c937e4267bcf3b176fb44d9e3. Jul 18 16:25:20.130545 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:20.130545 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 16:25:20.130545 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 16:25:20.130545 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 16:25:20.130545 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 16:25:20.130545 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 16:25:20.130545 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 16:25:20.130545 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 16:25:20.130545 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application conn.query(q) Jul 18 16:25:20.130545 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 16:25:20.130545 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 16:25:20.130545 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 16:25:20.131469 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application result.read() Jul 18 16:25:20.131469 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 16:25:20.131469 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 16:25:20.131469 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 16:25:20.131469 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 16:25:20.131469 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 16:25:20.131469 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 16:25:20.131469 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 16:25:20.131469 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 16:25:20.131469 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 18 16:25:20.131469 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:20.131469 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 18 16:25:20.131469 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:20.131469 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:20.131469 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 18 16:25:20.131469 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 18 16:25:20.131469 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 18 16:25:20.131469 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 18 16:25:20.132509 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 16:25:20.132509 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application next(self.gen) Jul 18 16:25:20.132509 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 18 16:25:20.132509 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application yield resource Jul 18 16:25:20.132509 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 16:25:20.132509 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application next(self.gen) Jul 18 16:25:20.132509 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 18 16:25:20.132509 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application self.session.rollback() Jul 18 16:25:20.132509 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 18 16:25:20.132509 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application self.force_reraise() Jul 18 16:25:20.132509 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 18 16:25:20.132509 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise self.value Jul 18 16:25:20.132509 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 18 16:25:20.132509 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 18 16:25:20.132509 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 18 16:25:20.132509 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application session.commit() Jul 18 16:25:20.132509 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 18 16:25:20.133428 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 18 16:25:20.133428 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 18 16:25:20.133428 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application self._prepare_impl() Jul 18 16:25:20.133428 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 18 16:25:20.133428 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application self.session.flush() Jul 18 16:25:20.133428 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 18 16:25:20.133428 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application self._flush(objects) Jul 18 16:25:20.133428 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 18 16:25:20.133428 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 18 16:25:20.133428 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 18 16:25:20.133428 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application compat.raise_( Jul 18 16:25:20.133428 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 16:25:20.133428 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception Jul 18 16:25:20.133428 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 18 16:25:20.133428 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application flush_context.execute() Jul 18 16:25:20.133428 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 18 16:25:20.133428 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rec.execute(self) Jul 18 16:25:20.135272 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 18 16:25:20.135272 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 18 16:25:20.135272 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 18 16:25:20.135272 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 18 16:25:20.135272 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 18 16:25:20.135272 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 18 16:25:20.135272 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 18 16:25:20.135272 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 18 16:25:20.135272 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 18 16:25:20.135272 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 18 16:25:20.135272 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 18 16:25:20.135272 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 18 16:25:20.135272 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 18 16:25:20.135272 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 18 16:25:20.135272 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 18 16:25:20.136222 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 18 16:25:20.136222 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 16:25:20.136222 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception Jul 18 16:25:20.136222 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 16:25:20.136222 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 16:25:20.136222 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 16:25:20.136222 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 16:25:20.136222 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 16:25:20.136222 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 16:25:20.136222 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 16:25:20.136222 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application conn.query(q) Jul 18 16:25:20.136222 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 16:25:20.136222 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 16:25:20.136222 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 16:25:20.136222 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application result.read() Jul 18 16:25:20.136222 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 16:25:20.136222 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 16:25:20.137121 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 16:25:20.137121 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 16:25:20.137121 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 16:25:20.137121 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 16:25:20.137121 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 16:25:20.137121 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 16:25:20.137121 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 18 16:25:20.137121 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 18 16:25:20.137121 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [parameters: {'id': '511d372c937e4267bcf3b176fb44d9e3', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 18 16:25:20.137121 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 18 16:25:20.137121 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:20.137121 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:25:20.137121 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:20.137121 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:20.137121 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 18 16:25:20.137914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 18 16:25:20.137914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 18 16:25:20.137914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 18 16:25:20.137914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 18 16:25:20.137914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:20.137914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:25:20.137914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:20.137914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:20.137914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:20.137914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:20.137914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:20.137914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:20.137914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:20.137914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:20.137914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:20.137914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:20.137914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:20.140655 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:20.140655 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 18 16:25:20.140655 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 18 16:25:20.140655 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:20.140655 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:20.140655 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 18 16:25:20.140655 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 18 16:25:20.140655 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 511d372c937e4267bcf3b176fb44d9e3. Jul 18 16:25:20.140655 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:20.140655 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 469/941] 10.222.0.26 () {60 vars in 1140 bytes} [Mon Jul 18 16:25:19 2022] POST /identity/v3/projects => generated 262 bytes in 271 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:25:20.142241 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9625f0b8-2b6f-4c26-a59e-3a2a1bea3f68 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:20.142413 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524b3fff-c9fd-4c13-a2f4-883d49b74729 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:20.193622 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f57de76-1e9f-423c-8109-61a7d53995be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:20.195063 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57de76-1e9f-423c-8109-61a7d53995be None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:20.195624 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57de76-1e9f-423c-8109-61a7d53995be None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:20.196112 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57de76-1e9f-423c-8109-61a7d53995be None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:20.197279 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f57de76-1e9f-423c-8109-61a7d53995be None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:20.198402 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f57de76-1e9f-423c-8109-61a7d53995be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:20.200435 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f57de76-1e9f-423c-8109-61a7d53995be None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:20.231538 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 473/942] 10.222.0.26 () {56 vars in 1149 bytes} [Mon Jul 18 16:25:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:20.233096 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f57de76-1e9f-423c-8109-61a7d53995be None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:20.256762 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:20.268322 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524b3fff-c9fd-4c13-a2f4-883d49b74729 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:20.280068 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:20.307485 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43b96ea-80b8-45ca-ac83-4f2ed5e6cdad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:20.308111 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b96ea-80b8-45ca-ac83-4f2ed5e6cdad None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:20.308390 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b96ea-80b8-45ca-ac83-4f2ed5e6cdad None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:20.308667 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b96ea-80b8-45ca-ac83-4f2ed5e6cdad None admin] PATH_INFO: `/v3/projects/service` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:20.320922 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43b96ea-80b8-45ca-ac83-4f2ed5e6cdad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:20.326230 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43b96ea-80b8-45ca-ac83-4f2ed5e6cdad None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:20.344773 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-d43b96ea-80b8-45ca-ac83-4f2ed5e6cdad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:25:20.344773 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:20.344773 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:20.344773 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:20.344773 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:20.344773 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:20.344773 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:20.344773 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:20.344773 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:20.344773 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:20.344773 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:20.344773 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:20.344773 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:25:20.344773 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:25:20.344773 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:25:20.345756 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:25:20.345756 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:20.345756 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:20.345756 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:20.345756 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:20.345756 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:20.345756 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:20.345756 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:20.345756 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:20.345756 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:20.345756 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:20.345756 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:20.345756 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:20.345756 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:20.345756 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:20.345756 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:20.345756 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:20.346644 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:25:20.346644 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:25:20.346644 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:25:20.346644 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:25:20.346644 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:25:20.346644 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:25:20.346644 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:25:20.346644 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:20.352332 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 470/943] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:25:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:20.353512 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43b96ea-80b8-45ca-ac83-4f2ed5e6cdad None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:20.383898 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f57de76-1e9f-423c-8109-61a7d53995be None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:20.423231 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ce56f3-0581-4c53-8f31-9242a9472cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:20.424328 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce56f3-0581-4c53-8f31-9242a9472cba None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:20.424713 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce56f3-0581-4c53-8f31-9242a9472cba None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:20.425332 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce56f3-0581-4c53-8f31-9242a9472cba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:20.436488 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ce56f3-0581-4c53-8f31-9242a9472cba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:20.438598 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ce56f3-0581-4c53-8f31-9242a9472cba None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:20.458312 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-e7ce56f3-0581-4c53-8f31-9242a9472cba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:20.458312 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:20.458312 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:20.458312 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:20.458312 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:20.458312 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:20.458312 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:20.458312 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:20.458312 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:20.458312 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:20.458312 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:20.458312 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:20.458312 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:25:20.458312 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:25:20.458312 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:25:20.459458 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:25:20.459458 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:20.459458 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:20.459458 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:20.459458 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:20.459458 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:20.459458 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:20.459458 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:20.459458 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:20.459458 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:20.459458 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:20.459458 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:20.459458 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:20.459458 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:20.459458 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:20.459458 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:20.459458 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:20.461265 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:25:20.461265 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:25:20.461265 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:20.461265 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:20.466711 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 474/944] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:20.467747 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ce56f3-0581-4c53-8f31-9242a9472cba None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:20.488591 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43b96ea-80b8-45ca-ac83-4f2ed5e6cdad None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:20.530899 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74725fd-dade-4718-8a0a-a2f8c4261c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:20.531676 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74725fd-dade-4718-8a0a-a2f8c4261c7e None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:20.532057 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74725fd-dade-4718-8a0a-a2f8c4261c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:20.532448 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74725fd-dade-4718-8a0a-a2f8c4261c7e None admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:20.533313 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74725fd-dade-4718-8a0a-a2f8c4261c7e None admin] RBAC: Adding query filter params (name=service) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:20.533720 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74725fd-dade-4718-8a0a-a2f8c4261c7e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:20.535561 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74725fd-dade-4718-8a0a-a2f8c4261c7e None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:20.546507 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:20.563785 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 471/945] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Jul 18 16:25:20 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:20.574969 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ea5613-9fc7-409a-9a75-4830c4ee8d3b None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:20.575250 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ea5613-9fc7-409a-9a75-4830c4ee8d3b None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:20.575567 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ea5613-9fc7-409a-9a75-4830c4ee8d3b None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:20.581568 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 472/946] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:25:20 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:25:20.583248 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec03742-6b7a-48dc-9f34-338edb1700c2 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:20.583412 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec03742-6b7a-48dc-9f34-338edb1700c2 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:20.583713 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec03742-6b7a-48dc-9f34-338edb1700c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:20.591718 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ce56f3-0581-4c53-8f31-9242a9472cba None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:20.603398 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-6ec03742-6b7a-48dc-9f34-338edb1700c2 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:20.604606 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-6ec03742-6b7a-48dc-9f34-338edb1700c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:20.631757 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:20.645763 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b102b499-a0fd-42a2-adc7-d1e35c63a3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:20.646957 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b102b499-a0fd-42a2-adc7-d1e35c63a3be None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:20.649311 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b102b499-a0fd-42a2-adc7-d1e35c63a3be None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:20.649610 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b102b499-a0fd-42a2-adc7-d1e35c63a3be None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:20.650412 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b102b499-a0fd-42a2-adc7-d1e35c63a3be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:20.651624 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b102b499-a0fd-42a2-adc7-d1e35c63a3be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:20.653368 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b102b499-a0fd-42a2-adc7-d1e35c63a3be None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:20.677185 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 475/947] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:25:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:20.680143 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b102b499-a0fd-42a2-adc7-d1e35c63a3be None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:20.708010 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 473/948] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:20 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:20.723369 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec03742-6b7a-48dc-9f34-338edb1700c2 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:20.816844 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b102b499-a0fd-42a2-adc7-d1e35c63a3be None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:20.853462 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:20.863573 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec03742-6b7a-48dc-9f34-338edb1700c2 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:20.864657 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aca861-e35d-4ae8-8ee8-f92efcfec2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:20.865468 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aca861-e35d-4ae8-8ee8-f92efcfec2d5 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:20.865749 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aca861-e35d-4ae8-8ee8-f92efcfec2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:20.869716 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aca861-e35d-4ae8-8ee8-f92efcfec2d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:20.883719 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aca861-e35d-4ae8-8ee8-f92efcfec2d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:20.885703 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aca861-e35d-4ae8-8ee8-f92efcfec2d5 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:20.907036 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-13aca861-e35d-4ae8-8ee8-f92efcfec2d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:25:20.907036 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:20.907036 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:20.907036 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:20.907036 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:20.907036 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:20.907036 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:20.907036 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:20.907036 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:20.907036 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:20.907036 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:20.907036 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:20.907036 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:25:20.907036 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:25:20.907036 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:25:20.908055 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:25:20.908055 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:20.908055 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:20.908055 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:20.908055 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:20.908055 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:20.908055 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:20.908055 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:20.908055 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:20.908055 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:20.908055 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:20.908055 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:20.908055 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:20.908055 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:20.908055 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:20.908055 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:20.908055 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:20.908944 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:25:20.908944 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:25:20.908944 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:25:20.908944 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:25:20.908944 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:25:20.908944 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:25:20.908944 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:25:20.908944 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:20.908944 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 476/949] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:25:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:25:20.913363 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187b2bfc-bf37-42e3-b17d-57a250bba7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:20.915061 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b2bfc-bf37-42e3-b17d-57a250bba7ad None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:20.915701 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b2bfc-bf37-42e3-b17d-57a250bba7ad None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:20.915928 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aca861-e35d-4ae8-8ee8-f92efcfec2d5 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:20.916445 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b2bfc-bf37-42e3-b17d-57a250bba7ad None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:20.917316 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187b2bfc-bf37-42e3-b17d-57a250bba7ad None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:20.919522 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187b2bfc-bf37-42e3-b17d-57a250bba7ad None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:20.933601 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-187b2bfc-bf37-42e3-b17d-57a250bba7ad None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 16:25:20.933601 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:20.933601 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:25:20.933601 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:25:20.933601 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:25:20.933601 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:25:20.933601 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:25:20.933601 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:20.933601 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 16:25:20.933601 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:20.933601 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:25:20.933601 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:20.933601 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:20.933601 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:20.933601 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:20.933601 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:20.933601 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:20.934894 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:20.934894 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:20.934894 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:20.934894 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:20.934894 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:20.934894 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:20.934894 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:25:20.934894 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:25:20.934894 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:25:20.934894 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:25:20.934894 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:20.934894 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:20.934894 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:20.934894 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:20.934894 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:20.934894 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:20.936069 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:20.936069 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:20.936069 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:20.936069 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:20.936069 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:20.936069 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:20.936069 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:20.936069 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:20.936069 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:20.936069 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:20.936069 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:25:20.936069 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:20.936069 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 16:25:20.936069 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:20.939214 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 474/950] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Jul 18 16:25:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:25:20.946316 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187b2bfc-bf37-42e3-b17d-57a250bba7ad None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:20.981450 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:21.052889 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aca861-e35d-4ae8-8ee8-f92efcfec2d5 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:21.089853 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187b2bfc-bf37-42e3-b17d-57a250bba7ad None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:21.104243 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd9d0c-be0e-4e20-beaa-23edbfb4c080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:21.105789 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd9d0c-be0e-4e20-beaa-23edbfb4c080 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:21.106582 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd9d0c-be0e-4e20-beaa-23edbfb4c080 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:21.107080 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd9d0c-be0e-4e20-beaa-23edbfb4c080 None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:21.108414 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bd9d0c-be0e-4e20-beaa-23edbfb4c080 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:21.109538 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bd9d0c-be0e-4e20-beaa-23edbfb4c080 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:21.111924 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bd9d0c-be0e-4e20-beaa-23edbfb4c080 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:21.135731 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 477/951] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 18 16:25:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:21.138888 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd9d0c-be0e-4e20-beaa-23edbfb4c080 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:21.142311 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49cff66-327d-42b9-90a9-ee0ba10692dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:21.143423 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49cff66-327d-42b9-90a9-ee0ba10692dd None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:21.143784 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49cff66-327d-42b9-90a9-ee0ba10692dd None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:21.144071 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49cff66-327d-42b9-90a9-ee0ba10692dd None admin] PATH_INFO: `/v3/services` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:21.144823 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49cff66-327d-42b9-90a9-ee0ba10692dd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:21.145134 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49cff66-327d-42b9-90a9-ee0ba10692dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:21.148543 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49cff66-327d-42b9-90a9-ee0ba10692dd None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:21.159610 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 475/952] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Jul 18 16:25:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:21.164415 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49cff66-327d-42b9-90a9-ee0ba10692dd None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:21.207514 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:21.269437 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd9d0c-be0e-4e20-beaa-23edbfb4c080 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:21.280045 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:21.301093 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49cff66-327d-42b9-90a9-ee0ba10692dd None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:21.309651 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d09f45b-ba77-4c5c-b7b2-96658604ea75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:21.311706 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09f45b-ba77-4c5c-b7b2-96658604ea75 None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:21.312207 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09f45b-ba77-4c5c-b7b2-96658604ea75 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:21.312665 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09f45b-ba77-4c5c-b7b2-96658604ea75 None admin] PATH_INFO: `/v3/projects/eb9747e2299c4e0991fd7edb4462c7e5/users/da799b0044f54da6835aa06e43ce1a6f/roles/3394397b06be42fe9507041df2452f79` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:21.315743 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d09f45b-ba77-4c5c-b7b2-96658604ea75 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb9747e2299c4e0991fd7edb4462c7e5, user_id=da799b0044f54da6835aa06e43ce1a6f, role_id=3394397b06be42fe9507041df2452f79)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:21.317627 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d09f45b-ba77-4c5c-b7b2-96658604ea75 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:21.340033 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303bbe7f-2856-4b53-83a6-592165d76c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:21.340624 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303bbe7f-2856-4b53-83a6-592165d76c8a None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:21.340906 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303bbe7f-2856-4b53-83a6-592165d76c8a None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:21.341178 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303bbe7f-2856-4b53-83a6-592165d76c8a None admin] PATH_INFO: `/v3/services` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:21.341887 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303bbe7f-2856-4b53-83a6-592165d76c8a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:21.342191 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303bbe7f-2856-4b53-83a6-592165d76c8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:21.343666 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303bbe7f-2856-4b53-83a6-592165d76c8a None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:21.354232 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 476/953] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Jul 18 16:25:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:21.355388 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303bbe7f-2856-4b53-83a6-592165d76c8a None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:21.395532 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 478/954] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 18 16:25:21 2022] PUT /identity/v3/projects/eb9747e2299c4e0991fd7edb4462c7e5/users/da799b0044f54da6835aa06e43ce1a6f/roles/3394397b06be42fe9507041df2452f79 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:25:21.482766 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303bbe7f-2856-4b53-83a6-592165d76c8a None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:21.521908 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb5e02-a4b9-48c4-9bb7-284d1cd0e37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:21.522637 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb5e02-a4b9-48c4-9bb7-284d1cd0e37c None admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:21.523438 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb5e02-a4b9-48c4-9bb7-284d1cd0e37c None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:21.523873 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb5e02-a4b9-48c4-9bb7-284d1cd0e37c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:21.524980 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cb5e02-a4b9-48c4-9bb7-284d1cd0e37c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:21.526504 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cb5e02-a4b9-48c4-9bb7-284d1cd0e37c None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:21.602021 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 477/955] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Jul 18 16:25:21 2022] POST /identity/v3/endpoints => generated 349 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:25:21.895082 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff78ac-af4b-43af-80bf-35c5e5273245 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:21.895648 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff78ac-af4b-43af-80bf-35c5e5273245 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:21.895648 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff78ac-af4b-43af-80bf-35c5e5273245 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:21.897126 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 479/956] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:25:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:21.903952 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da601f16-58a4-4494-996c-a89058e64fe0 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:21.904582 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da601f16-58a4-4494-996c-a89058e64fe0 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:21.905071 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da601f16-58a4-4494-996c-a89058e64fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:21.917788 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:21.923555 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-da601f16-58a4-4494-996c-a89058e64fe0 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:21.925586 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-da601f16-58a4-4494-996c-a89058e64fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:22.014219 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 478/957] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:21 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:22.036027 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ff78ac-af4b-43af-80bf-35c5e5273245 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:22.147907 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ff78ac-af4b-43af-80bf-35c5e5273245 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:22.155828 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:22.196092 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5842ef9-ef75-413f-808d-2e6caebead67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:22.197650 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5842ef9-ef75-413f-808d-2e6caebead67 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:22.197948 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5842ef9-ef75-413f-808d-2e6caebead67 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:22.198237 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5842ef9-ef75-413f-808d-2e6caebead67 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:22.198998 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5842ef9-ef75-413f-808d-2e6caebead67 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:22.200801 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5842ef9-ef75-413f-808d-2e6caebead67 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:22.214317 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-e5842ef9-ef75-413f-808d-2e6caebead67 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 16:25:22.214317 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:22.214317 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:25:22.214317 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:25:22.214317 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:25:22.214317 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:25:22.214317 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:25:22.214317 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:22.214317 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 16:25:22.214317 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:22.214317 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:25:22.214317 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:22.214317 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:22.214317 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:22.214317 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:22.214317 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:22.214317 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:22.215707 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:22.215707 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:22.215707 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:22.215707 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:22.215707 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:22.215707 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:22.215707 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:25:22.215707 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:25:22.215707 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:25:22.215707 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:25:22.215707 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:22.215707 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:22.215707 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:22.215707 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:22.215707 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:22.215707 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:22.216659 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:22.216659 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:22.216659 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:22.216659 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:22.216659 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:22.216659 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:22.216659 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:22.216659 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:22.216659 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:22.216659 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:22.216659 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:25:22.216659 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:22.216659 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 16:25:22.216659 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:22.216659 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 480/958] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:25:22 2022] GET /identity/v3/services/compute => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:22.224637 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da601f16-58a4-4494-996c-a89058e64fe0 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:22.340808 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da601f16-58a4-4494-996c-a89058e64fe0 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:22.373945 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:22.378831 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35f01c0-0afb-4214-ba3a-4c6b17853ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:22.379561 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f01c0-0afb-4214-ba3a-4c6b17853ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:22.379854 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f01c0-0afb-4214-ba3a-4c6b17853ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:22.380135 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f01c0-0afb-4214-ba3a-4c6b17853ad8 None admin] PATH_INFO: `/v3/services` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:22.380846 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35f01c0-0afb-4214-ba3a-4c6b17853ad8 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:22.381154 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35f01c0-0afb-4214-ba3a-4c6b17853ad8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:22.382631 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35f01c0-0afb-4214-ba3a-4c6b17853ad8 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:22.388197 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 479/959] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Jul 18 16:25:22 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:22.394490 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5842ef9-ef75-413f-808d-2e6caebead67 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:22.511441 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5842ef9-ef75-413f-808d-2e6caebead67 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:22.549288 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826d7ac8-b805-4ed9-981b-12f61c925a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:22.550332 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d7ac8-b805-4ed9-981b-12f61c925a07 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:22.550822 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d7ac8-b805-4ed9-981b-12f61c925a07 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:22.551330 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d7ac8-b805-4ed9-981b-12f61c925a07 None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:22.552282 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826d7ac8-b805-4ed9-981b-12f61c925a07 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:22.552805 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826d7ac8-b805-4ed9-981b-12f61c925a07 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:22.554410 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826d7ac8-b805-4ed9-981b-12f61c925a07 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:22.559947 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 481/960] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Jul 18 16:25:22 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:22.773994 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b205775-5824-4088-9e06-d333dfeac86b None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:22.774230 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b205775-5824-4088-9e06-d333dfeac86b None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:22.774605 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b205775-5824-4088-9e06-d333dfeac86b None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:22.776161 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 480/961] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:25:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:22.781770 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d78a88-6e68-4d59-a161-6d381ec88f3f None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:22.782082 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d78a88-6e68-4d59-a161-6d381ec88f3f None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:22.782345 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d78a88-6e68-4d59-a161-6d381ec88f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:22.800386 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-48d78a88-6e68-4d59-a161-6d381ec88f3f None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:22.801580 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-48d78a88-6e68-4d59-a161-6d381ec88f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:22.831636 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:22.879601 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 482/962] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:25:22.900786 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b205775-5824-4088-9e06-d333dfeac86b None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:23.016155 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b205775-5824-4088-9e06-d333dfeac86b None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:23.054746 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1952c5d6-f46a-4540-acf9-388ad618d7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:23.055424 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1952c5d6-f46a-4540-acf9-388ad618d7d0 None admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:23.055737 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1952c5d6-f46a-4540-acf9-388ad618d7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:23.056015 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1952c5d6-f46a-4540-acf9-388ad618d7d0 None admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:23.056738 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1952c5d6-f46a-4540-acf9-388ad618d7d0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:23.058174 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1952c5d6-f46a-4540-acf9-388ad618d7d0 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:23.120179 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.sql.core [None req-1952c5d6-f46a-4540-acf9-388ad618d7d0 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 18 16:25:23.120179 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 18 16:25:23.120179 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [parameters: {'user_id': '931320c2c1cd4187a45a8195778317fa', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 18 16:25:23.120179 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80367) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 18 16:25:23.128058 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-1952c5d6-f46a-4540-acf9-388ad618d7d0 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 18 16:25:23.128058 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:23.128058 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 16:25:23.128058 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 16:25:23.128058 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 16:25:23.128058 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 16:25:23.128058 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 16:25:23.128058 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 16:25:23.128058 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 16:25:23.128058 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application conn.query(q) Jul 18 16:25:23.128058 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 16:25:23.128058 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 16:25:23.128058 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 16:25:23.128058 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application result.read() Jul 18 16:25:23.128058 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 16:25:23.129177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 16:25:23.129177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 16:25:23.129177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 16:25:23.129177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 16:25:23.129177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 16:25:23.129177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 16:25:23.129177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 16:25:23.129177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 18 16:25:23.129177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:23.129177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 18 16:25:23.129177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:23.129177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:23.129177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 18 16:25:23.129177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 18 16:25:23.129177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 18 16:25:23.129177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 18 16:25:23.129177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 16:25:23.129177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application next(self.gen) Jul 18 16:25:23.130588 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 18 16:25:23.130588 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application yield resource Jul 18 16:25:23.130588 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 16:25:23.130588 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application next(self.gen) Jul 18 16:25:23.130588 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 18 16:25:23.130588 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application self.session.rollback() Jul 18 16:25:23.130588 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 18 16:25:23.130588 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application self.force_reraise() Jul 18 16:25:23.130588 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 18 16:25:23.130588 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise self.value Jul 18 16:25:23.130588 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 18 16:25:23.130588 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 18 16:25:23.130588 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 18 16:25:23.130588 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application session.commit() Jul 18 16:25:23.130588 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 18 16:25:23.130588 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 18 16:25:23.130588 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 18 16:25:23.131836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application self._prepare_impl() Jul 18 16:25:23.131836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 18 16:25:23.131836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application self.session.flush() Jul 18 16:25:23.131836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 18 16:25:23.131836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application self._flush(objects) Jul 18 16:25:23.131836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 18 16:25:23.131836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 18 16:25:23.131836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 18 16:25:23.131836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application compat.raise_( Jul 18 16:25:23.131836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 16:25:23.131836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception Jul 18 16:25:23.131836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 18 16:25:23.131836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application flush_context.execute() Jul 18 16:25:23.131836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 18 16:25:23.131836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rec.execute(self) Jul 18 16:25:23.131836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 18 16:25:23.131836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 18 16:25:23.134231 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 18 16:25:23.134231 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 18 16:25:23.134231 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 18 16:25:23.134231 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 18 16:25:23.134231 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 18 16:25:23.134231 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 18 16:25:23.134231 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 18 16:25:23.134231 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 18 16:25:23.134231 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 18 16:25:23.134231 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 18 16:25:23.134231 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 18 16:25:23.134231 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 18 16:25:23.134231 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 18 16:25:23.134231 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 18 16:25:23.134231 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 16:25:23.134231 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception Jul 18 16:25:23.136935 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 16:25:23.136935 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 16:25:23.136935 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 16:25:23.136935 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 16:25:23.136935 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 16:25:23.136935 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 16:25:23.136935 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 16:25:23.136935 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application conn.query(q) Jul 18 16:25:23.136935 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 16:25:23.136935 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 16:25:23.136935 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 16:25:23.136935 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application result.read() Jul 18 16:25:23.136935 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 16:25:23.136935 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 16:25:23.136935 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 16:25:23.136935 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 16:25:23.139267 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 16:25:23.139267 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 16:25:23.139267 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 16:25:23.139267 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 16:25:23.139267 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 18 16:25:23.139267 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 18 16:25:23.139267 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [parameters: {'user_id': '931320c2c1cd4187a45a8195778317fa', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 18 16:25:23.139267 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 18 16:25:23.139267 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:23.139267 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:25:23.139267 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:23.139267 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:23.139267 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:23.139267 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:23.139267 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:23.142294 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:23.142294 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:23.142294 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:23.142294 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:23.142294 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:23.142294 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:23.142294 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:23.142294 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 18 16:25:23.142294 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 18 16:25:23.142294 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:23.142294 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:23.142294 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:25:23.142294 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:23.142294 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:25:23.142294 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:23.142294 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 18 16:25:23.142294 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 18 16:25:23.144514 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 18 16:25:23.144514 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 18 16:25:23.144514 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 18 16:25:23.144514 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 18 16:25:23.144514 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 18 16:25:23.144514 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:23.144514 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 481/963] 10.222.0.26 () {60 vars in 1131 bytes} [Mon Jul 18 16:25:22 2022] POST /identity/v3/users => generated 160 bytes in 229 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:23.145414 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d78a88-6e68-4d59-a161-6d381ec88f3f None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:23.248676 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d78a88-6e68-4d59-a161-6d381ec88f3f None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:23.286933 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180bf785-971d-454e-951c-d8ea82618348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:23.288247 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180bf785-971d-454e-951c-d8ea82618348 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:23.288760 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180bf785-971d-454e-951c-d8ea82618348 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:23.289243 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180bf785-971d-454e-951c-d8ea82618348 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:23.297806 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180bf785-971d-454e-951c-d8ea82618348 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:23.299641 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180bf785-971d-454e-951c-d8ea82618348 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:23.316498 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-180bf785-971d-454e-951c-d8ea82618348 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 16:25:23.316498 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:23.316498 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:23.316498 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:23.316498 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:23.316498 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:23.316498 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:23.316498 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:23.316498 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:23.316498 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:23.316498 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:23.316498 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:23.316498 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:25:23.316498 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:25:23.316498 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:25:23.316498 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:25:23.317492 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:23.317492 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:23.317492 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:25:23.317492 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:23.317492 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:25:23.317492 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:23.317492 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:23.317492 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:23.317492 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:23.317492 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:23.317492 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:23.317492 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:23.317492 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:23.317492 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:23.317492 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:23.317492 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:23.317492 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:23.317492 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:23.318558 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:23.318558 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:23.318558 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:25:23.318558 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:25:23.318558 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:25:23.318558 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:25:23.318558 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:25:23.318558 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:25:23.318558 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 16:25:23.318558 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:23.322021 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 483/964] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:25:23 2022] GET /identity/v3/users/cinder => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:23.327820 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1952c5d6-f46a-4540-acf9-388ad618d7d0 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:23.398135 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:23.445230 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1952c5d6-f46a-4540-acf9-388ad618d7d0 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:23.485392 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b65cce-3c35-45ec-9489-ea1b534326bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:23.486079 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b65cce-3c35-45ec-9489-ea1b534326bb None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:23.486368 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b65cce-3c35-45ec-9489-ea1b534326bb None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:23.486645 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b65cce-3c35-45ec-9489-ea1b534326bb None admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:23.487455 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b65cce-3c35-45ec-9489-ea1b534326bb None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:23.487784 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b65cce-3c35-45ec-9489-ea1b534326bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:23.489418 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b65cce-3c35-45ec-9489-ea1b534326bb None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:23.530280 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 482/965] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 18 16:25:23 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:23.662474 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf024f8-8162-49b4-83d6-dd6c520e1f14 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:23.662673 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf024f8-8162-49b4-83d6-dd6c520e1f14 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:23.662932 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf024f8-8162-49b4-83d6-dd6c520e1f14 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:23.665317 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 484/966] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:25:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:23.671589 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01795ee0-4720-43a5-b920-1e4970ab4aa5 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:23.671893 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01795ee0-4720-43a5-b920-1e4970ab4aa5 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:23.672215 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01795ee0-4720-43a5-b920-1e4970ab4aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:23.691413 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-01795ee0-4720-43a5-b920-1e4970ab4aa5 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:23.692998 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-01795ee0-4720-43a5-b920-1e4970ab4aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:23.768958 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 483/967] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:25:23.793504 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf024f8-8162-49b4-83d6-dd6c520e1f14 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:23.826136 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:23.914353 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf024f8-8162-49b4-83d6-dd6c520e1f14 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:23.926488 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4dcdba-b3c7-4506-9b1e-5f77e8e41728 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:23.927017 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4dcdba-b3c7-4506-9b1e-5f77e8e41728 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:23.927372 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4dcdba-b3c7-4506-9b1e-5f77e8e41728 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:23.929099 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 484/968] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:25:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:23.934399 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ad6d4-312f-414c-83f8-047c320beac9 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:23.934789 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ad6d4-312f-414c-83f8-047c320beac9 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:23.935144 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ad6d4-312f-414c-83f8-047c320beac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:23.953042 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dc0ab5-88ec-4ce1-9877-0f21a992f1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:23.953858 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc0ab5-88ec-4ce1-9877-0f21a992f1ff None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:23.954141 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc0ab5-88ec-4ce1-9877-0f21a992f1ff None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:23.954420 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc0ab5-88ec-4ce1-9877-0f21a992f1ff None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:23.955862 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-546ad6d4-312f-414c-83f8-047c320beac9 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:23.957276 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-546ad6d4-312f-414c-83f8-047c320beac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:23.966218 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dc0ab5-88ec-4ce1-9877-0f21a992f1ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:25:23.974535 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-77dc0ab5-88ec-4ce1-9877-0f21a992f1ff None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 16:25:23.974535 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:23.974535 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:23.974535 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:23.974535 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:23.974535 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:23.974535 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:23.974535 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:23.974535 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:23.974535 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:23.974535 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:23.974535 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:23.974535 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:25:23.974535 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:25:23.974535 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:25:23.974535 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:25:23.975524 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:25:23.975524 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:25:23.975524 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:25:23.975524 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:25:23.975524 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:23.975524 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:23.975524 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:23.975524 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:23.975524 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:23.975524 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:23.975524 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:23.975524 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:23.975524 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:23.975524 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:23.975524 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:23.975524 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:23.977302 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:23.977302 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:23.977302 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:23.977302 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:23.977302 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:25:23.977302 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:25:23.977302 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:25:23.977302 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:25:23.977302 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:25:23.977302 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:25:23.977302 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 16:25:23.977302 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:23.977302 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 485/969] 10.222.0.26 () {56 vars in 1120 bytes} [Mon Jul 18 16:25:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:23.982288 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dc0ab5-88ec-4ce1-9877-0f21a992f1ff None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:24.010470 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:24.037900 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 485/970] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:25:24.058765 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546ad6d4-312f-414c-83f8-047c320beac9 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:24.078485 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:24.096096 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dc0ab5-88ec-4ce1-9877-0f21a992f1ff None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:24.136716 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ffef69-58bc-4db5-9ca1-e25da8bed182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:24.137391 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ffef69-58bc-4db5-9ca1-e25da8bed182 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:24.137732 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ffef69-58bc-4db5-9ca1-e25da8bed182 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:24.138024 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ffef69-58bc-4db5-9ca1-e25da8bed182 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:24.138765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ffef69-58bc-4db5-9ca1-e25da8bed182 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:24.139080 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ffef69-58bc-4db5-9ca1-e25da8bed182 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:24.140592 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ffef69-58bc-4db5-9ca1-e25da8bed182 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:24.150693 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 486/971] 10.222.0.26 () {56 vars in 1115 bytes} [Mon Jul 18 16:25:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:24.156131 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ffef69-58bc-4db5-9ca1-e25da8bed182 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:24.168453 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546ad6d4-312f-414c-83f8-047c320beac9 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:24.209256 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0ab34a-08ad-4c53-9e37-601c9e9181af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:24.210337 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ab34a-08ad-4c53-9e37-601c9e9181af None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:24.210673 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ab34a-08ad-4c53-9e37-601c9e9181af None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:24.210953 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ab34a-08ad-4c53-9e37-601c9e9181af None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:24.211644 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0ab34a-08ad-4c53-9e37-601c9e9181af None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:24.213041 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0ab34a-08ad-4c53-9e37-601c9e9181af None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:24.226994 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-de0ab34a-08ad-4c53-9e37-601c9e9181af None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 16:25:24.226994 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:24.226994 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:25:24.226994 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:25:24.226994 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:25:24.226994 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:25:24.226994 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:25:24.226994 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:24.226994 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 16:25:24.226994 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:24.226994 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:25:24.226994 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:24.226994 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:24.226994 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:24.226994 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:24.226994 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:24.226994 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:24.228934 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:24.228934 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:24.228934 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:24.228934 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:24.228934 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:24.228934 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:24.228934 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:25:24.228934 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:25:24.228934 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:25:24.228934 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:25:24.228934 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:24.228934 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:24.228934 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:24.228934 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:24.228934 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:24.228934 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:24.229889 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:24.229889 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:24.229889 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:24.229889 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:24.229889 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:24.229889 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:24.229889 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:24.229889 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:24.229889 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:24.229889 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:24.229889 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:25:24.229889 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:24.229889 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 16:25:24.229889 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:24.229889 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 486/972] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Jul 18 16:25:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:25:24.234375 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0ab34a-08ad-4c53-9e37-601c9e9181af None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:24.281927 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ffef69-58bc-4db5-9ca1-e25da8bed182 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:24.284709 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:24.321964 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560c3c98-09b5-49ac-af15-b1240ee5adae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:24.322771 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c3c98-09b5-49ac-af15-b1240ee5adae None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:24.323057 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c3c98-09b5-49ac-af15-b1240ee5adae None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:24.323334 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c3c98-09b5-49ac-af15-b1240ee5adae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:24.335795 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560c3c98-09b5-49ac-af15-b1240ee5adae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:24.337264 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560c3c98-09b5-49ac-af15-b1240ee5adae None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:24.345024 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:24.354386 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0ab34a-08ad-4c53-9e37-601c9e9181af None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:24.357459 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-560c3c98-09b5-49ac-af15-b1240ee5adae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:24.357459 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:24.357459 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:24.357459 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:24.357459 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:24.357459 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:24.357459 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:24.357459 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:24.357459 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:24.357459 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:24.357459 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:24.357459 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:24.357459 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:25:24.357459 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:25:24.357459 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:25:24.360606 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:25:24.360606 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:24.360606 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:24.360606 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:24.360606 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:24.360606 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:24.360606 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:24.360606 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:24.360606 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:24.360606 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:24.360606 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:24.360606 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:24.360606 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:24.360606 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:24.360606 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:24.360606 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:24.360606 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:24.361714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:25:24.361714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:25:24.361714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:24.361714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:24.363335 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 487/973] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:25:24.364384 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560c3c98-09b5-49ac-af15-b1240ee5adae None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:24.396725 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf147b36-635c-4f7f-8360-39c869b0ddb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:24.397666 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf147b36-635c-4f7f-8360-39c869b0ddb8 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:24.398115 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf147b36-635c-4f7f-8360-39c869b0ddb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:24.398447 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf147b36-635c-4f7f-8360-39c869b0ddb8 None admin] PATH_INFO: `/v3/services` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:24.399354 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf147b36-635c-4f7f-8360-39c869b0ddb8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:24.399697 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf147b36-635c-4f7f-8360-39c869b0ddb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:24.401075 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf147b36-635c-4f7f-8360-39c869b0ddb8 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:24.406469 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 487/974] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Jul 18 16:25:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:24.412153 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf147b36-635c-4f7f-8360-39c869b0ddb8 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:24.477378 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560c3c98-09b5-49ac-af15-b1240ee5adae None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:24.513292 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c5925e-e878-47e9-a178-faa6587a86a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:24.514068 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5925e-e878-47e9-a178-faa6587a86a0 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:24.514346 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5925e-e878-47e9-a178-faa6587a86a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:24.514608 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5925e-e878-47e9-a178-faa6587a86a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:24.515325 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c5925e-e878-47e9-a178-faa6587a86a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:24.515658 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c5925e-e878-47e9-a178-faa6587a86a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:24.517249 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c5925e-e878-47e9-a178-faa6587a86a0 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:24.533592 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 488/975] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:25:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:24.533776 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf147b36-635c-4f7f-8360-39c869b0ddb8 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:24.537095 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:24.539669 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c5925e-e878-47e9-a178-faa6587a86a0 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:24.576200 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7893195c-20f0-4327-97a9-3cb70469c0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:24.577532 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893195c-20f0-4327-97a9-3cb70469c0c3 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:24.578545 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893195c-20f0-4327-97a9-3cb70469c0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:24.578831 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893195c-20f0-4327-97a9-3cb70469c0c3 None admin] PATH_INFO: `/v3/services` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:24.579939 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7893195c-20f0-4327-97a9-3cb70469c0c3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:24.580252 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7893195c-20f0-4327-97a9-3cb70469c0c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:24.581645 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7893195c-20f0-4327-97a9-3cb70469c0c3 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:24.592706 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 488/976] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Jul 18 16:25:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:24.594350 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7893195c-20f0-4327-97a9-3cb70469c0c3 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:24.614540 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:24.672619 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c5925e-e878-47e9-a178-faa6587a86a0 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:24.717295 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1532a87e-8aea-499e-8d83-2de3e0bc11cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:24.718104 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1532a87e-8aea-499e-8d83-2de3e0bc11cd None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:24.718383 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1532a87e-8aea-499e-8d83-2de3e0bc11cd None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:24.718679 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1532a87e-8aea-499e-8d83-2de3e0bc11cd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:24.720351 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7893195c-20f0-4327-97a9-3cb70469c0c3 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:24.727154 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1532a87e-8aea-499e-8d83-2de3e0bc11cd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:24.728674 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1532a87e-8aea-499e-8d83-2de3e0bc11cd None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:24.744533 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-1532a87e-8aea-499e-8d83-2de3e0bc11cd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 16:25:24.744533 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:24.744533 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:24.744533 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:24.744533 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:24.744533 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:24.744533 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:24.744533 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:24.744533 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:24.744533 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:24.744533 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:24.744533 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:24.744533 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:25:24.744533 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:25:24.744533 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:25:24.744533 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:25:24.745756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:24.745756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:24.745756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:25:24.745756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:24.745756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:25:24.745756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:24.745756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:24.745756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:24.745756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:24.745756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:24.745756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:24.745756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:24.745756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:24.745756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:24.745756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:24.745756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:24.745756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:24.745756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:24.751652 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:24.751652 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:24.751652 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:25:24.751652 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:25:24.751652 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:25:24.751652 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:25:24.751652 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:25:24.751652 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:25:24.751652 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 16:25:24.751652 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:24.751652 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 489/977] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Jul 18 16:25:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:24.751652 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8db41-ae16-4c9c-a8f0-f66aaa3ec5cd None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:24.751652 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8db41-ae16-4c9c-a8f0-f66aaa3ec5cd None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:24.752425 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8db41-ae16-4c9c-a8f0-f66aaa3ec5cd None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:24.752425 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 490/978] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:25:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:24.752425 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc8db41-ae16-4c9c-a8f0-f66aaa3ec5cd None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:24.777656 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017f23d6-2ce5-45f8-b4c4-b2723c7a57dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:24.778321 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f23d6-2ce5-45f8-b4c4-b2723c7a57dd None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:24.778607 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f23d6-2ce5-45f8-b4c4-b2723c7a57dd None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:24.778880 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f23d6-2ce5-45f8-b4c4-b2723c7a57dd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:24.779708 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017f23d6-2ce5-45f8-b4c4-b2723c7a57dd None admin] RBAC: Adding query filter params (interface=admin, service_id=53fcca302b884a29b289944126fa2d24, region_id=RegionOne) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:24.780019 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017f23d6-2ce5-45f8-b4c4-b2723c7a57dd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:24.781379 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017f23d6-2ce5-45f8-b4c4-b2723c7a57dd None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:24.786798 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 489/979] 10.222.0.26 () {56 vars in 1249 bytes} [Mon Jul 18 16:25:24 2022] GET /identity/v3/endpoints?service_id=53fcca302b884a29b289944126fa2d24&interface=admin®ion_id=RegionOne => generated 193 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:24.788669 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307c89b-7ce3-4213-8dd4-462457c2c05d None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:24.788961 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307c89b-7ce3-4213-8dd4-462457c2c05d None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:24.789239 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307c89b-7ce3-4213-8dd4-462457c2c05d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:24.805783 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-7307c89b-7ce3-4213-8dd4-462457c2c05d None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:24.806980 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-7307c89b-7ce3-4213-8dd4-462457c2c05d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:24.839232 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:24.870998 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc8db41-ae16-4c9c-a8f0-f66aaa3ec5cd None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:24.882207 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 490/980] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:24.883066 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7307c89b-7ce3-4213-8dd4-462457c2c05d None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:24.890655 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:24.910235 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6ab93c-2673-4b0a-9492-beb14703acda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:24.912582 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6ab93c-2673-4b0a-9492-beb14703acda None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:24.912582 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6ab93c-2673-4b0a-9492-beb14703acda None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:24.912582 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6ab93c-2673-4b0a-9492-beb14703acda None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:24.912582 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6ab93c-2673-4b0a-9492-beb14703acda None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:24.912582 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6ab93c-2673-4b0a-9492-beb14703acda None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:24.914149 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6ab93c-2673-4b0a-9492-beb14703acda None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:24.931856 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 491/981] 10.222.0.26 () {56 vars in 1149 bytes} [Mon Jul 18 16:25:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:24.932680 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6ab93c-2673-4b0a-9492-beb14703acda None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:24.992298 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7307c89b-7ce3-4213-8dd4-462457c2c05d None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:25.027402 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6251d55b-c729-41a3-9ac7-7bb146bce4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:25.028112 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251d55b-c729-41a3-9ac7-7bb146bce4ad None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:25.028400 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251d55b-c729-41a3-9ac7-7bb146bce4ad None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:25.028682 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251d55b-c729-41a3-9ac7-7bb146bce4ad None admin] PATH_INFO: `/v3/services` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:25.029368 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6251d55b-c729-41a3-9ac7-7bb146bce4ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:25.030761 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6251d55b-c729-41a3-9ac7-7bb146bce4ad None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:25.036525 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6ab93c-2673-4b0a-9492-beb14703acda None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:25.037071 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 491/982] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Jul 18 16:25:24 2022] GET /identity/v3/services => generated 1497 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:25:25.037985 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6251d55b-c729-41a3-9ac7-7bb146bce4ad None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:25.071731 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:25.092539 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a94fef9-63b4-49ca-986d-41713c61f364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:25.093413 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a94fef9-63b4-49ca-986d-41713c61f364 None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:25.093773 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a94fef9-63b4-49ca-986d-41713c61f364 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:25.094056 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a94fef9-63b4-49ca-986d-41713c61f364 None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:25.094810 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a94fef9-63b4-49ca-986d-41713c61f364 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:25.096270 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a94fef9-63b4-49ca-986d-41713c61f364 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:25.157322 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 492/983] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jul 18 16:25:24 2022] POST /identity/v3/services => generated 242 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:25:25.163313 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6251d55b-c729-41a3-9ac7-7bb146bce4ad None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:25.218975 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a788061-df31-479a-8493-2991f485dadf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:25.219863 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a788061-df31-479a-8493-2991f485dadf None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:25.220251 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a788061-df31-479a-8493-2991f485dadf None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:25.220623 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a788061-df31-479a-8493-2991f485dadf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:25.304739 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a788061-df31-479a-8493-2991f485dadf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:25.306240 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a788061-df31-479a-8493-2991f485dadf None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:25.333622 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:25.412796 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 492/984] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:25:25 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 16:25:25.419057 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a94fef9-63b4-49ca-986d-41713c61f364 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:25.460559 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:25.533086 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a94fef9-63b4-49ca-986d-41713c61f364 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:25.568600 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e8c7d2-b7b7-4d88-87c5-11355a5a316d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:25.569989 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8c7d2-b7b7-4d88-87c5-11355a5a316d None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:25.570275 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8c7d2-b7b7-4d88-87c5-11355a5a316d None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:25.570551 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8c7d2-b7b7-4d88-87c5-11355a5a316d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:25.581961 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e8c7d2-b7b7-4d88-87c5-11355a5a316d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:25.583420 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e8c7d2-b7b7-4d88-87c5-11355a5a316d None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:25.601837 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-12e8c7d2-b7b7-4d88-87c5-11355a5a316d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:25.601837 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:25.601837 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:25.601837 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:25.601837 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:25.601837 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:25.601837 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:25.601837 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:25.601837 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:25.601837 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:25.601837 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:25.601837 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:25.601837 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:25:25.601837 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:25:25.601837 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:25:25.602862 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:25:25.602862 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:25.602862 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:25.602862 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:25.602862 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:25.602862 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:25.602862 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:25.602862 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:25.602862 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:25.602862 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:25.602862 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:25.602862 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:25.602862 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:25.602862 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:25.602862 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:25.602862 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:25.602862 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:25.608719 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:25:25.608719 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:25:25.608719 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:25.608719 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:25.608719 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 493/985] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:25.609017 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a788061-df31-479a-8493-2991f485dadf None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:25.719951 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a788061-df31-479a-8493-2991f485dadf None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:25.747109 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:25.759549 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f72f4e-7f22-45e2-a810-aadc0effd0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:25.760221 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f72f4e-7f22-45e2-a810-aadc0effd0f0 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:25.760544 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f72f4e-7f22-45e2-a810-aadc0effd0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:25.760829 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f72f4e-7f22-45e2-a810-aadc0effd0f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:25.761520 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f72f4e-7f22-45e2-a810-aadc0effd0f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:25.761823 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f72f4e-7f22-45e2-a810-aadc0effd0f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:25.763189 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f72f4e-7f22-45e2-a810-aadc0effd0f0 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:25.780675 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 493/986] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:25:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:25.782033 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e8c7d2-b7b7-4d88-87c5-11355a5a316d None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:25.888167 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:25.901330 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e8c7d2-b7b7-4d88-87c5-11355a5a316d None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:25.940528 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8466f418-67a7-47f2-8ef7-9f74b67e517b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:25.941391 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466f418-67a7-47f2-8ef7-9f74b67e517b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:25.941677 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466f418-67a7-47f2-8ef7-9f74b67e517b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:25.941987 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466f418-67a7-47f2-8ef7-9f74b67e517b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:25.949445 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8466f418-67a7-47f2-8ef7-9f74b67e517b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:25.950900 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8466f418-67a7-47f2-8ef7-9f74b67e517b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:25.966163 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-8466f418-67a7-47f2-8ef7-9f74b67e517b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:25:25.966163 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:25.966163 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:25.966163 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:25.966163 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:25.966163 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:25.966163 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:25.966163 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:25.966163 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:25.966163 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:25.966163 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:25.966163 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:25.966163 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:25:25.966163 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:25:25.966163 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:25:25.967073 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:25:25.967073 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:25.967073 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:25.967073 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:25.967073 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:25.967073 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:25.967073 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:25.967073 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:25.967073 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:25.967073 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:25.967073 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:25.967073 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:25.967073 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:25.967073 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:25.967073 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:25.967073 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:25.967073 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:25.968346 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:25:25.968346 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:25:25.968346 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:25:25.968346 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:25:25.968346 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:25:25.968346 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:25:25.968346 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:25:25.968346 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:25.969104 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 494/987] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:25:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:25.973343 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f72f4e-7f22-45e2-a810-aadc0effd0f0 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:26.078017 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f72f4e-7f22-45e2-a810-aadc0effd0f0 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:26.115609 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f73c186-3642-49dd-9339-e3dd8eee5637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:26.116285 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73c186-3642-49dd-9339-e3dd8eee5637 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:26.116571 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73c186-3642-49dd-9339-e3dd8eee5637 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:26.116942 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73c186-3642-49dd-9339-e3dd8eee5637 None admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:26.117644 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f73c186-3642-49dd-9339-e3dd8eee5637 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:26.117955 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f73c186-3642-49dd-9339-e3dd8eee5637 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:26.119391 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f73c186-3642-49dd-9339-e3dd8eee5637 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:26.136425 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 494/988] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 18 16:25:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:26.137739 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8466f418-67a7-47f2-8ef7-9f74b67e517b None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:26.268502 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8466f418-67a7-47f2-8ef7-9f74b67e517b None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:26.306048 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537cb704-b745-47d9-9f8e-d8586f76cb03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:26.306724 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537cb704-b745-47d9-9f8e-d8586f76cb03 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:26.307088 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537cb704-b745-47d9-9f8e-d8586f76cb03 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:26.307538 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537cb704-b745-47d9-9f8e-d8586f76cb03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:26.308443 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537cb704-b745-47d9-9f8e-d8586f76cb03 None admin] RBAC: Adding query filter params (role.id=3394397b06be42fe9507041df2452f79, scope.project.id=eb9747e2299c4e0991fd7edb4462c7e5, user.id=da799b0044f54da6835aa06e43ce1a6f) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:26.308760 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537cb704-b745-47d9-9f8e-d8586f76cb03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:26.310401 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537cb704-b745-47d9-9f8e-d8586f76cb03 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:26.317526 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 495/989] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 18 16:25:26 2022] GET /identity/v3/role_assignments?user.id=da799b0044f54da6835aa06e43ce1a6f&scope.project.id=eb9747e2299c4e0991fd7edb4462c7e5&role.id=3394397b06be42fe9507041df2452f79 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:27.145686 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc78a5d-034c-4c8c-83ef-9224dc8e23be None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:27.146187 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc78a5d-034c-4c8c-83ef-9224dc8e23be None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:27.146187 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc78a5d-034c-4c8c-83ef-9224dc8e23be None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:27.149464 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 495/990] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:25:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:27.154373 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd67213-358a-40a0-9b25-1c995b24491e None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:27.154668 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd67213-358a-40a0-9b25-1c995b24491e None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:27.154949 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd67213-358a-40a0-9b25-1c995b24491e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:27.174243 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-7dd67213-358a-40a0-9b25-1c995b24491e None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:27.175442 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-7dd67213-358a-40a0-9b25-1c995b24491e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:27.184804 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:27.246640 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccccf39-4670-4ab5-b42f-d7283352c8de None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:27.246924 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccccf39-4670-4ab5-b42f-d7283352c8de None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:27.247203 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccccf39-4670-4ab5-b42f-d7283352c8de None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:27.248884 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 496/991] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:25:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:25:27.255709 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc99db6-3755-4a97-8d9b-9a342a4a137c None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:27.256239 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc99db6-3755-4a97-8d9b-9a342a4a137c None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:27.256239 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc99db6-3755-4a97-8d9b-9a342a4a137c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:27.258476 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 496/992] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:27 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:27.274703 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-3dc99db6-3755-4a97-8d9b-9a342a4a137c None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:27.278943 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-3dc99db6-3755-4a97-8d9b-9a342a4a137c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:27.281503 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd67213-358a-40a0-9b25-1c995b24491e None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:27.316318 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:27.350086 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 497/993] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:27 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:27.373479 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc99db6-3755-4a97-8d9b-9a342a4a137c None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:27.388376 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd67213-358a-40a0-9b25-1c995b24491e None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:27.435696 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b9390f-08cc-4095-827f-a90356de5cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:27.435696 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b9390f-08cc-4095-827f-a90356de5cdf None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:27.435696 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b9390f-08cc-4095-827f-a90356de5cdf None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:27.435696 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b9390f-08cc-4095-827f-a90356de5cdf None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:27.436175 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b9390f-08cc-4095-827f-a90356de5cdf None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:27.436175 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b9390f-08cc-4095-827f-a90356de5cdf None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:27.440318 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-47b9390f-08cc-4095-827f-a90356de5cdf None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 16:25:27.440318 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:27.440318 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:25:27.440318 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:25:27.440318 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:25:27.440318 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:25:27.440318 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:25:27.440318 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:27.440318 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 16:25:27.440318 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:27.440318 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:25:27.440318 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:27.440318 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:27.440318 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:27.440318 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:27.440318 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:27.440318 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:27.441282 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:27.441282 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:27.441282 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:27.441282 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:27.441282 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:27.441282 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:27.441282 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:25:27.441282 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:25:27.441282 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:25:27.441282 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:25:27.441282 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:27.441282 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:27.441282 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:27.441282 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:27.441282 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:27.441282 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:27.442117 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:27.442117 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:27.442117 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:27.442117 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:27.442117 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:27.442117 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:27.442117 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:27.442117 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:27.442117 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:27.442117 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:27.442117 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:25:27.442117 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:27.442117 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 16:25:27.442117 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:27.447288 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 497/994] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Jul 18 16:25:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:25:27.447979 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b9390f-08cc-4095-827f-a90356de5cdf None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:27.458334 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:27.490179 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc99db6-3755-4a97-8d9b-9a342a4a137c None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:27.532566 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8b66b8-9285-4849-8eeb-e36392fff8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:27.533379 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b66b8-9285-4849-8eeb-e36392fff8a9 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:27.533662 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b66b8-9285-4849-8eeb-e36392fff8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:27.534400 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b66b8-9285-4849-8eeb-e36392fff8a9 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:27.535077 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8b66b8-9285-4849-8eeb-e36392fff8a9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:27.536870 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8b66b8-9285-4849-8eeb-e36392fff8a9 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:27.553082 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-9f8b66b8-9285-4849-8eeb-e36392fff8a9 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 16:25:27.553082 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:27.553082 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:25:27.553082 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:25:27.553082 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:25:27.553082 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:25:27.553082 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:25:27.553082 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:27.553082 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 16:25:27.553082 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:27.553082 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:25:27.553082 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:27.553082 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:27.553082 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:27.553082 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:27.553082 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:27.553082 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:27.554396 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:27.554396 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:27.554396 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:27.554396 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:27.554396 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:27.554396 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:27.554396 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:25:27.554396 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:25:27.554396 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:25:27.554396 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:25:27.554396 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:27.554396 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:27.554396 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:27.554396 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:27.554396 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:27.554396 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:27.555230 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:27.555230 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:27.555230 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:27.555230 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:27.555230 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:27.555230 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:27.555230 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:27.555230 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:27.555230 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:27.555230 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:27.555230 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:25:27.555230 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:27.555230 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 16:25:27.555230 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:27.556006 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 498/995] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:25:27 2022] GET /identity/v3/services/compute => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:27.561022 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8b66b8-9285-4849-8eeb-e36392fff8a9 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:27.561455 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b9390f-08cc-4095-827f-a90356de5cdf None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:27.602906 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb608996-041e-4a1b-ae1e-882af04ce772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:27.603569 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb608996-041e-4a1b-ae1e-882af04ce772 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:27.603854 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb608996-041e-4a1b-ae1e-882af04ce772 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:27.604129 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb608996-041e-4a1b-ae1e-882af04ce772 None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:27.604835 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb608996-041e-4a1b-ae1e-882af04ce772 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:27.605138 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb608996-041e-4a1b-ae1e-882af04ce772 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:27.606404 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:27.606692 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb608996-041e-4a1b-ae1e-882af04ce772 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:27.612997 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 498/996] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Jul 18 16:25:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:27.619259 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb608996-041e-4a1b-ae1e-882af04ce772 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:27.673918 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8b66b8-9285-4849-8eeb-e36392fff8a9 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:27.698723 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:27.710493 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e53aad-3a88-4746-a94c-4308209ced46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:27.711173 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e53aad-3a88-4746-a94c-4308209ced46 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:27.711459 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e53aad-3a88-4746-a94c-4308209ced46 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:27.711771 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e53aad-3a88-4746-a94c-4308209ced46 None admin] PATH_INFO: `/v3/services` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:27.712478 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e53aad-3a88-4746-a94c-4308209ced46 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:27.712787 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e53aad-3a88-4746-a94c-4308209ced46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:27.714159 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e53aad-3a88-4746-a94c-4308209ced46 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:27.723950 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 499/997] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Jul 18 16:25:27 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:27.724906 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e53aad-3a88-4746-a94c-4308209ced46 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:27.744336 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb608996-041e-4a1b-ae1e-882af04ce772 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:27.778780 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545b15e2-e4bf-4861-9a2c-14a0d4c87e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:27.779451 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b15e2-e4bf-4861-9a2c-14a0d4c87e33 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:27.779785 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b15e2-e4bf-4861-9a2c-14a0d4c87e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:27.780070 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b15e2-e4bf-4861-9a2c-14a0d4c87e33 None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:27.780930 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545b15e2-e4bf-4861-9a2c-14a0d4c87e33 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:27.781245 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545b15e2-e4bf-4861-9a2c-14a0d4c87e33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:27.782621 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545b15e2-e4bf-4861-9a2c-14a0d4c87e33 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:27.793079 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 499/998] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Jul 18 16:25:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:27.794824 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545b15e2-e4bf-4861-9a2c-14a0d4c87e33 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:27.838676 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e53aad-3a88-4746-a94c-4308209ced46 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:27.884004 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c522d9-76b9-408d-931a-ec88d1fcb43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:27.896715 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:27.896715 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c522d9-76b9-408d-931a-ec88d1fcb43e None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:27.896715 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c522d9-76b9-408d-931a-ec88d1fcb43e None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:27.896715 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c522d9-76b9-408d-931a-ec88d1fcb43e None admin] PATH_INFO: `/v3/services` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:27.896715 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c522d9-76b9-408d-931a-ec88d1fcb43e None admin] RBAC: Adding query filter params (type=compute) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:27.896715 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c522d9-76b9-408d-931a-ec88d1fcb43e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:27.897873 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c522d9-76b9-408d-931a-ec88d1fcb43e None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:27.897873 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 500/999] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Jul 18 16:25:27 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:27.901350 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c522d9-76b9-408d-931a-ec88d1fcb43e None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:27.910956 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545b15e2-e4bf-4861-9a2c-14a0d4c87e33 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:27.948254 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a88faef-8245-41b1-8709-b5fe27f720cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:27.952056 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a88faef-8245-41b1-8709-b5fe27f720cc None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:27.952056 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a88faef-8245-41b1-8709-b5fe27f720cc None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:27.952056 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a88faef-8245-41b1-8709-b5fe27f720cc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:27.952056 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a88faef-8245-41b1-8709-b5fe27f720cc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:27.952056 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a88faef-8245-41b1-8709-b5fe27f720cc None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:27.984934 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:28.003820 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c522d9-76b9-408d-931a-ec88d1fcb43e None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:28.038334 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d00839-c0ea-4ac9-868c-ba45fbde47dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:28.039131 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d00839-c0ea-4ac9-868c-ba45fbde47dc None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:28.039419 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d00839-c0ea-4ac9-868c-ba45fbde47dc None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:28.039728 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d00839-c0ea-4ac9-868c-ba45fbde47dc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:28.040529 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d00839-c0ea-4ac9-868c-ba45fbde47dc None admin] RBAC: Adding query filter params (interface=public, service_id=7cb84a4bf5464615b1d40520c3780396, region_id=RegionOne) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:28.040841 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d00839-c0ea-4ac9-868c-ba45fbde47dc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:28.042499 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d00839-c0ea-4ac9-868c-ba45fbde47dc None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:28.048042 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 501/1000] 10.222.0.26 () {56 vars in 1251 bytes} [Mon Jul 18 16:25:27 2022] GET /identity/v3/endpoints?service_id=7cb84a4bf5464615b1d40520c3780396&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:28.054323 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d00839-c0ea-4ac9-868c-ba45fbde47dc None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:28.110048 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 500/1001] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Jul 18 16:25:27 2022] POST /identity/v3/endpoints => generated 325 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:25:28.132202 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:28.176527 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d00839-c0ea-4ac9-868c-ba45fbde47dc None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:28.225730 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3e1b80-f428-4993-97d7-3bef111c846a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:28.226809 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e1b80-f428-4993-97d7-3bef111c846a None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:28.227095 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e1b80-f428-4993-97d7-3bef111c846a None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:28.227373 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e1b80-f428-4993-97d7-3bef111c846a None admin] PATH_INFO: `/v3/services` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:28.228107 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3e1b80-f428-4993-97d7-3bef111c846a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:28.229525 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3e1b80-f428-4993-97d7-3bef111c846a None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:28.244178 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 502/1002] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Jul 18 16:25:28 2022] GET /identity/v3/services => generated 1727 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:25:28.452611 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021667e8-16cb-4f15-bbf8-776d650e1c94 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:28.453130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021667e8-16cb-4f15-bbf8-776d650e1c94 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:28.453208 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021667e8-16cb-4f15-bbf8-776d650e1c94 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:28.454723 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 501/1003] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:25:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:28.460015 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e92c707-f1ba-4711-9f43-8543a2070b52 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:28.460295 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e92c707-f1ba-4711-9f43-8543a2070b52 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:28.460582 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e92c707-f1ba-4711-9f43-8543a2070b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:28.477596 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-3e92c707-f1ba-4711-9f43-8543a2070b52 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:28.478714 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-3e92c707-f1ba-4711-9f43-8543a2070b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:28.555787 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 503/1004] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:28 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:28.578222 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021667e8-16cb-4f15-bbf8-776d650e1c94 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:28.686882 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021667e8-16cb-4f15-bbf8-776d650e1c94 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:28.722226 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee498ae3-0baa-4eed-869c-ae1f87351c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:28.723021 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee498ae3-0baa-4eed-869c-ae1f87351c3f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:28.723305 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee498ae3-0baa-4eed-869c-ae1f87351c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:28.723604 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee498ae3-0baa-4eed-869c-ae1f87351c3f None admin] PATH_INFO: `/v3/services/image` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:28.724265 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee498ae3-0baa-4eed-869c-ae1f87351c3f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:28.726173 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee498ae3-0baa-4eed-869c-ae1f87351c3f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:28.738427 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-ee498ae3-0baa-4eed-869c-ae1f87351c3f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 16:25:28.738427 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:28.738427 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:25:28.738427 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:25:28.738427 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:25:28.738427 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:25:28.738427 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:25:28.738427 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:28.738427 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 16:25:28.738427 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:28.738427 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:25:28.738427 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:28.738427 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:28.738427 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:28.738427 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:28.738427 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:28.738427 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:28.739831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:28.739831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:28.739831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:28.739831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:28.739831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:28.739831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:28.739831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:25:28.739831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:25:28.739831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:25:28.739831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:25:28.739831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:28.739831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:28.739831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:28.739831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:28.739831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:28.739831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:28.741638 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:28.741638 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:28.741638 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:28.741638 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:28.741638 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:28.741638 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:28.741638 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:28.741638 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:28.741638 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:28.741638 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:28.741638 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:25:28.741638 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:28.741638 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 16:25:28.741638 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:28.743856 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 502/1005] 10.222.0.26 () {56 vars in 1105 bytes} [Mon Jul 18 16:25:28 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:28.746568 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e92c707-f1ba-4711-9f43-8543a2070b52 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:28.765069 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:28.861373 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e92c707-f1ba-4711-9f43-8543a2070b52 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:28.900829 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b10d39-966f-4c06-a880-c235a89c1dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:28.901502 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b10d39-966f-4c06-a880-c235a89c1dce None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:28.901785 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b10d39-966f-4c06-a880-c235a89c1dce None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:28.902060 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b10d39-966f-4c06-a880-c235a89c1dce None admin] PATH_INFO: `/v3/services` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:28.902766 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b10d39-966f-4c06-a880-c235a89c1dce None admin] RBAC: Adding query filter params (name=image) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:28.903067 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b10d39-966f-4c06-a880-c235a89c1dce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:28.904461 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b10d39-966f-4c06-a880-c235a89c1dce None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:28.914823 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 504/1006] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:28 2022] GET /identity/v3/services?name=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:28.916484 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee498ae3-0baa-4eed-869c-ae1f87351c3f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:28.978336 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:29.020859 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee498ae3-0baa-4eed-869c-ae1f87351c3f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:29.056387 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa13233b-9756-45e1-9cf8-e9b2487924e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:29.057058 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13233b-9756-45e1-9cf8-e9b2487924e8 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:29.057342 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13233b-9756-45e1-9cf8-e9b2487924e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:29.057620 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13233b-9756-45e1-9cf8-e9b2487924e8 None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:29.058319 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa13233b-9756-45e1-9cf8-e9b2487924e8 None admin] RBAC: Adding query filter params (type=image) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:29.058626 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa13233b-9756-45e1-9cf8-e9b2487924e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:29.060092 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa13233b-9756-45e1-9cf8-e9b2487924e8 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:29.071849 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 503/1007] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:28 2022] GET /identity/v3/services?type=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:30.238996 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f953be-18c1-426a-ac71-88b39e17cb84 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:30.239567 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f953be-18c1-426a-ac71-88b39e17cb84 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:30.239567 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f953be-18c1-426a-ac71-88b39e17cb84 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:30.241487 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 505/1008] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:25:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:30.247117 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d20c58-9df0-4340-8990-e4a20bb63cb7 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:30.247615 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d20c58-9df0-4340-8990-e4a20bb63cb7 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:30.247911 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d20c58-9df0-4340-8990-e4a20bb63cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:30.265368 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-36d20c58-9df0-4340-8990-e4a20bb63cb7 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:30.266560 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-36d20c58-9df0-4340-8990-e4a20bb63cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:30.304985 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2177b96-c046-4a9f-9931-59391917337f None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:30.305336 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2177b96-c046-4a9f-9931-59391917337f None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:30.305546 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2177b96-c046-4a9f-9931-59391917337f None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:30.307535 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 506/1009] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:25:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:30.313221 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4ff6c0-f8a2-4555-b6b7-8ad25e14860d None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:30.313522 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4ff6c0-f8a2-4555-b6b7-8ad25e14860d None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:30.313775 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4ff6c0-f8a2-4555-b6b7-8ad25e14860d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:30.331799 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-7b4ff6c0-f8a2-4555-b6b7-8ad25e14860d None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:30.332998 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-7b4ff6c0-f8a2-4555-b6b7-8ad25e14860d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:30.341062 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 504/1010] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:30 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:30.364619 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d20c58-9df0-4340-8990-e4a20bb63cb7 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:30.409353 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 507/1011] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:30 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:25:30.424875 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:30.433106 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4ff6c0-f8a2-4555-b6b7-8ad25e14860d None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:30.453644 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:30.478329 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d20c58-9df0-4340-8990-e4a20bb63cb7 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:30.518551 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16565512-2f7e-45d9-bde3-d64ccdd23fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:30.519713 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16565512-2f7e-45d9-bde3-d64ccdd23fc6 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:30.519844 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16565512-2f7e-45d9-bde3-d64ccdd23fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:30.520046 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16565512-2f7e-45d9-bde3-d64ccdd23fc6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:30.522241 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16565512-2f7e-45d9-bde3-d64ccdd23fc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:30.524081 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16565512-2f7e-45d9-bde3-d64ccdd23fc6 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:30.525971 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 505/1012] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:30.531559 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16565512-2f7e-45d9-bde3-d64ccdd23fc6 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:30.548664 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4ff6c0-f8a2-4555-b6b7-8ad25e14860d None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:30.585545 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea3f20-b1e1-40aa-a6c8-b93e401da272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:30.586354 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea3f20-b1e1-40aa-a6c8-b93e401da272 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:30.586665 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea3f20-b1e1-40aa-a6c8-b93e401da272 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:30.586944 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea3f20-b1e1-40aa-a6c8-b93e401da272 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:30.587631 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea3f20-b1e1-40aa-a6c8-b93e401da272 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:30.589007 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea3f20-b1e1-40aa-a6c8-b93e401da272 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:30.601688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-faea3f20-b1e1-40aa-a6c8-b93e401da272 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 16:25:30.601688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:30.601688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:25:30.601688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:25:30.601688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:25:30.601688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:25:30.601688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:25:30.601688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:30.601688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 16:25:30.601688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:30.601688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:25:30.601688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:30.601688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:30.601688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:30.601688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:30.601688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:30.601688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:30.602753 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:30.602753 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:30.602753 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:30.602753 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:30.602753 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:30.602753 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:30.602753 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:25:30.602753 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:25:30.602753 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:25:30.602753 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:25:30.602753 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:30.602753 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:30.602753 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:30.602753 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:30.602753 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:30.602753 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:30.603661 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:30.603661 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:30.603661 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:30.603661 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:30.603661 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:30.603661 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:30.603661 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:30.603661 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:30.603661 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:30.603661 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:30.603661 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:25:30.603661 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:30.603661 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 16:25:30.603661 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:30.603661 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 508/1013] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:25:30 2022] GET /identity/v3/services/compute => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:25:30.609002 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea3f20-b1e1-40aa-a6c8-b93e401da272 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:30.646110 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16565512-2f7e-45d9-bde3-d64ccdd23fc6 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:30.668704 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:30.683792 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd449ca-eaa4-44a7-8250-0b614b554b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:30.684457 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd449ca-eaa4-44a7-8250-0b614b554b00 None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:30.684778 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd449ca-eaa4-44a7-8250-0b614b554b00 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:30.685011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd449ca-eaa4-44a7-8250-0b614b554b00 None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:30.685723 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd449ca-eaa4-44a7-8250-0b614b554b00 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:30.687194 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd449ca-eaa4-44a7-8250-0b614b554b00 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:30.694734 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:30.718024 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea3f20-b1e1-40aa-a6c8-b93e401da272 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:30.752092 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d478e867-3986-4562-b85b-6689c70be1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:30.752812 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d478e867-3986-4562-b85b-6689c70be1a0 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:30.753098 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d478e867-3986-4562-b85b-6689c70be1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:30.753377 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d478e867-3986-4562-b85b-6689c70be1a0 None admin] PATH_INFO: `/v3/services` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:30.754088 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d478e867-3986-4562-b85b-6689c70be1a0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:30.754396 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d478e867-3986-4562-b85b-6689c70be1a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:30.755806 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d478e867-3986-4562-b85b-6689c70be1a0 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:30.761708 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 509/1014] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Jul 18 16:25:30 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:30.767402 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d478e867-3986-4562-b85b-6689c70be1a0 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:30.788172 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 506/1015] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jul 18 16:25:30 2022] POST /identity/v3/projects => generated 311 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:25:30.894330 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d478e867-3986-4562-b85b-6689c70be1a0 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:30.932570 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da320e25-c65f-4ae6-84f2-ffbf21709cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:30.933267 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da320e25-c65f-4ae6-84f2-ffbf21709cf8 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:30.933553 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da320e25-c65f-4ae6-84f2-ffbf21709cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:30.933830 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da320e25-c65f-4ae6-84f2-ffbf21709cf8 None admin] PATH_INFO: `/v3/services` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:30.934552 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da320e25-c65f-4ae6-84f2-ffbf21709cf8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:30.934856 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da320e25-c65f-4ae6-84f2-ffbf21709cf8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:30.936289 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da320e25-c65f-4ae6-84f2-ffbf21709cf8 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:30.942068 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 510/1016] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Jul 18 16:25:30 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:30.948752 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd449ca-eaa4-44a7-8250-0b614b554b00 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:31.057734 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd449ca-eaa4-44a7-8250-0b614b554b00 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:31.092226 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a392e7e-54d6-4b06-960a-11e49f076254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:31.092899 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a392e7e-54d6-4b06-960a-11e49f076254 None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:31.093184 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a392e7e-54d6-4b06-960a-11e49f076254 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:31.093459 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a392e7e-54d6-4b06-960a-11e49f076254 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:31.094220 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a392e7e-54d6-4b06-960a-11e49f076254 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:31.095671 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a392e7e-54d6-4b06-960a-11e49f076254 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:31.126253 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae666ba3-f582-4266-8893-36e0ac9bd4f3 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:31.126539 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae666ba3-f582-4266-8893-36e0ac9bd4f3 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:31.126847 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae666ba3-f582-4266-8893-36e0ac9bd4f3 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:31.128317 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 511/1017] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:25:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:25:31.133289 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969cc673-be12-4690-af51-ad03469aee87 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:31.133491 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969cc673-be12-4690-af51-ad03469aee87 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:31.133775 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969cc673-be12-4690-af51-ad03469aee87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:31.139961 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:31.151801 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-969cc673-be12-4690-af51-ad03469aee87 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:31.153001 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-969cc673-be12-4690-af51-ad03469aee87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:31.197718 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 507/1018] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Jul 18 16:25:30 2022] POST /identity/v3/endpoints => generated 334 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:25:31.238534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 512/1019] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:31 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:25:31.262728 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a392e7e-54d6-4b06-960a-11e49f076254 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:31.296548 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:31.376315 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a392e7e-54d6-4b06-960a-11e49f076254 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:31.415311 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b953662-5339-4871-9f0a-6995e1dd5eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:31.416045 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b953662-5339-4871-9f0a-6995e1dd5eca None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:31.416333 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b953662-5339-4871-9f0a-6995e1dd5eca None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:31.416864 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b953662-5339-4871-9f0a-6995e1dd5eca None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:31.417596 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b953662-5339-4871-9f0a-6995e1dd5eca None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:31.419032 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b953662-5339-4871-9f0a-6995e1dd5eca None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:31.523945 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 508/1020] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jul 18 16:25:31 2022] POST /identity/v3/services => generated 242 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:25:32.915141 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b7e280-98f3-4d2e-8ba6-7f5e60e53170 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:32.915700 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b7e280-98f3-4d2e-8ba6-7f5e60e53170 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:32.915812 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b7e280-98f3-4d2e-8ba6-7f5e60e53170 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:32.917508 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 513/1021] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:25:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:32.922604 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60727957-8d3f-4923-be18-cf8f8afab0ba None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:32.922991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60727957-8d3f-4923-be18-cf8f8afab0ba None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:32.923159 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60727957-8d3f-4923-be18-cf8f8afab0ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:32.940522 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-60727957-8d3f-4923-be18-cf8f8afab0ba None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:32.941715 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-60727957-8d3f-4923-be18-cf8f8afab0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:33.017665 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 509/1022] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:32 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:33.043573 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b7e280-98f3-4d2e-8ba6-7f5e60e53170 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:33.145554 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b7e280-98f3-4d2e-8ba6-7f5e60e53170 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:33.175603 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:33.182906 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9890437a-dd94-4b7f-a27c-758eeb87edf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:33.187983 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9890437a-dd94-4b7f-a27c-758eeb87edf3 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:33.187983 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9890437a-dd94-4b7f-a27c-758eeb87edf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:33.187983 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9890437a-dd94-4b7f-a27c-758eeb87edf3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:33.187983 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9890437a-dd94-4b7f-a27c-758eeb87edf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:33.187983 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9890437a-dd94-4b7f-a27c-758eeb87edf3 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:33.193052 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 514/1023] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:33.194236 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60727957-8d3f-4923-be18-cf8f8afab0ba None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:33.209699 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:33.297946 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60727957-8d3f-4923-be18-cf8f8afab0ba None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:33.333453 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eb5b00-1cff-4ef6-b8d7-410b99d92511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:33.334202 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb5b00-1cff-4ef6-b8d7-410b99d92511 None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:33.334488 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb5b00-1cff-4ef6-b8d7-410b99d92511 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:33.334966 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb5b00-1cff-4ef6-b8d7-410b99d92511 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:33.335726 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eb5b00-1cff-4ef6-b8d7-410b99d92511 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:33.337170 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eb5b00-1cff-4ef6-b8d7-410b99d92511 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:33.392893 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f4fe6-acfd-4565-b37f-b8e880a5e6df None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:33.393177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f4fe6-acfd-4565-b37f-b8e880a5e6df None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:33.393453 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f4fe6-acfd-4565-b37f-b8e880a5e6df None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:33.395307 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 515/1024] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:25:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:25:33.401206 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b783f-f4a3-431f-88b2-b219435c8da3 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:33.401519 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b783f-f4a3-431f-88b2-b219435c8da3 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:33.401880 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b783f-f4a3-431f-88b2-b219435c8da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:33.419683 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-386b783f-f4a3-431f-88b2-b219435c8da3 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:33.420877 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-386b783f-f4a3-431f-88b2-b219435c8da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:33.431660 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 510/1025] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 18 16:25:33 2022] POST /identity/v3/users => generated 285 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:25:33.508022 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 516/1026] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:33 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:25:33.531687 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eb5b00-1cff-4ef6-b8d7-410b99d92511 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:33.595292 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f834fd0-8444-4b8c-97ff-ecda41ee5eab None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:33.595626 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f834fd0-8444-4b8c-97ff-ecda41ee5eab None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:33.595901 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f834fd0-8444-4b8c-97ff-ecda41ee5eab None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:33.597490 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 517/1027] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:25:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:25:33.603206 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ab897-0356-485b-b517-633fb7e524d0 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:33.604182 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ab897-0356-485b-b517-633fb7e524d0 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:33.604182 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ab897-0356-485b-b517-633fb7e524d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:33.612914 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:33.623110 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-e41ab897-0356-485b-b517-633fb7e524d0 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:33.624340 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-e41ab897-0356-485b-b517-633fb7e524d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:33.653606 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eb5b00-1cff-4ef6-b8d7-410b99d92511 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:33.695265 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1635dd-f4c6-4320-8a3c-76a8f8abcd4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:33.696197 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1635dd-f4c6-4320-8a3c-76a8f8abcd4f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:33.696480 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1635dd-f4c6-4320-8a3c-76a8f8abcd4f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:33.696763 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1635dd-f4c6-4320-8a3c-76a8f8abcd4f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:33.708271 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1635dd-f4c6-4320-8a3c-76a8f8abcd4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:25:33.719539 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 518/1028] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:33 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:25:33.719714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-7d1635dd-f4c6-4320-8a3c-76a8f8abcd4f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 16:25:33.719714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:33.719714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:33.719714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:33.719714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:33.719714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:33.719714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:33.719714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:33.719714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:33.719714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:33.719714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:33.719714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:33.719714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:25:33.719714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:25:33.719714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:25:33.719714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:25:33.720792 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:25:33.720792 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:25:33.720792 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:25:33.720792 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:25:33.720792 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:33.720792 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:33.720792 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:33.720792 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:33.720792 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:33.720792 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:33.720792 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:33.720792 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:33.720792 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:33.720792 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:33.720792 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:33.720792 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:33.721630 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:33.721630 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:33.721630 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:33.721630 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:33.721630 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:25:33.721630 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:25:33.721630 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:25:33.721630 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:25:33.721630 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:25:33.721630 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:25:33.721630 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 16:25:33.721630 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:33.721630 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 511/1029] 10.222.0.26 () {56 vars in 1120 bytes} [Mon Jul 18 16:25:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:33.725675 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41ab897-0356-485b-b517-633fb7e524d0 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:33.742862 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1635dd-f4c6-4320-8a3c-76a8f8abcd4f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:33.798454 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:33.851698 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1635dd-f4c6-4320-8a3c-76a8f8abcd4f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:33.854798 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41ab897-0356-485b-b517-633fb7e524d0 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:33.883685 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:33.889661 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d2e6c5-1b9e-4502-a04b-2d385597ba99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:33.890463 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2e6c5-1b9e-4502-a04b-2d385597ba99 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:33.895537 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2e6c5-1b9e-4502-a04b-2d385597ba99 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:33.895537 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2e6c5-1b9e-4502-a04b-2d385597ba99 None admin] PATH_INFO: `/v3/services/image` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:33.895537 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d2e6c5-1b9e-4502-a04b-2d385597ba99 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:33.895537 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d2e6c5-1b9e-4502-a04b-2d385597ba99 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:33.895847 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a8bbc-91c7-46b5-95aa-368a3670d10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:33.895847 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a8bbc-91c7-46b5-95aa-368a3670d10f None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:33.895847 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a8bbc-91c7-46b5-95aa-368a3670d10f None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:33.895847 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a8bbc-91c7-46b5-95aa-368a3670d10f None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:33.896960 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29a8bbc-91c7-46b5-95aa-368a3670d10f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:33.897266 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29a8bbc-91c7-46b5-95aa-368a3670d10f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:33.898706 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29a8bbc-91c7-46b5-95aa-368a3670d10f None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:33.907089 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-49d2e6c5-1b9e-4502-a04b-2d385597ba99 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 16:25:33.907089 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:33.907089 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:25:33.907089 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:25:33.907089 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:25:33.907089 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:25:33.907089 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:25:33.907089 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:33.907089 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 16:25:33.907089 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:33.907089 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:25:33.907089 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:33.907089 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:33.907089 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:33.907089 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:33.907089 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:33.907089 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:33.908289 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:33.908289 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:33.908289 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:33.908289 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:33.908289 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:33.908289 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:33.908289 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:25:33.908289 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:25:33.908289 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:25:33.908289 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:25:33.908289 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:33.908289 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:33.908289 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:33.908289 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:33.908289 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:33.908289 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:33.909602 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:33.909602 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:33.909602 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:33.909602 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:33.909602 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:33.909602 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:33.909602 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:33.909602 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:33.909602 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:33.909602 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:33.909602 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:25:33.909602 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:33.909602 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 16:25:33.909602 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:33.910680 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 512/1030] 10.222.0.26 () {56 vars in 1105 bytes} [Mon Jul 18 16:25:33 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:33.911910 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 519/1031] 10.222.0.26 () {56 vars in 1115 bytes} [Mon Jul 18 16:25:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:33.914076 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d2e6c5-1b9e-4502-a04b-2d385597ba99 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:33.915333 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a8bbc-91c7-46b5-95aa-368a3670d10f None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:34.033807 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d2e6c5-1b9e-4502-a04b-2d385597ba99 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:34.038371 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a8bbc-91c7-46b5-95aa-368a3670d10f None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:34.061515 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:34.072886 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b533e-e61f-411f-bc36-096e9299f28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:34.073677 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b533e-e61f-411f-bc36-096e9299f28a None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:34.073959 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b533e-e61f-411f-bc36-096e9299f28a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:34.074231 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b533e-e61f-411f-bc36-096e9299f28a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:34.078581 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632c7539-be08-4105-b75e-271c4cf32e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:34.079598 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c7539-be08-4105-b75e-271c4cf32e02 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:34.080091 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c7539-be08-4105-b75e-271c4cf32e02 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:34.080589 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c7539-be08-4105-b75e-271c4cf32e02 None admin] PATH_INFO: `/v3/services` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:34.081517 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632c7539-be08-4105-b75e-271c4cf32e02 None admin] RBAC: Adding query filter params (name=image) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:34.081897 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632c7539-be08-4105-b75e-271c4cf32e02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:34.083274 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632c7539-be08-4105-b75e-271c4cf32e02 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:34.086580 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871b533e-e61f-411f-bc36-096e9299f28a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:34.088540 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871b533e-e61f-411f-bc36-096e9299f28a None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:34.094107 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 520/1032] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:33 2022] GET /identity/v3/services?name=image => generated 122 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:34.094924 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632c7539-be08-4105-b75e-271c4cf32e02 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:34.111062 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-871b533e-e61f-411f-bc36-096e9299f28a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:34.111062 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:34.111062 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:34.111062 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:34.111062 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:34.111062 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:34.111062 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:34.111062 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:34.111062 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:34.111062 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:34.111062 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:34.111062 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:34.111062 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:25:34.111062 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:25:34.111062 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:25:34.112092 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:25:34.112092 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:34.112092 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:34.112092 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:34.112092 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:34.112092 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:34.112092 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:34.112092 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:34.112092 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:34.112092 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:34.112092 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:34.112092 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:34.112092 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:34.112092 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:34.112092 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:34.112092 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:34.112092 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:34.118023 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:25:34.118023 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:25:34.118023 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:34.118023 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:34.118023 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 513/1033] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:34.118890 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b533e-e61f-411f-bc36-096e9299f28a None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:34.171181 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:34.213300 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632c7539-be08-4105-b75e-271c4cf32e02 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:34.237453 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b533e-e61f-411f-bc36-096e9299f28a None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:34.250739 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e45dded-03d2-4ff3-935c-82e86e1cb711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:34.251411 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45dded-03d2-4ff3-935c-82e86e1cb711 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:34.251720 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45dded-03d2-4ff3-935c-82e86e1cb711 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:34.252000 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45dded-03d2-4ff3-935c-82e86e1cb711 None admin] PATH_INFO: `/v3/services` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:34.252878 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e45dded-03d2-4ff3-935c-82e86e1cb711 None admin] RBAC: Adding query filter params (type=image) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:34.253148 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e45dded-03d2-4ff3-935c-82e86e1cb711 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:34.254517 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e45dded-03d2-4ff3-935c-82e86e1cb711 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:34.261613 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 521/1034] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:34 2022] GET /identity/v3/services?type=image => generated 350 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:34.269856 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e45dded-03d2-4ff3-935c-82e86e1cb711 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:34.277237 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cba527a-8b48-4cef-a452-1f7f657c2c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:34.277917 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba527a-8b48-4cef-a452-1f7f657c2c9e None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:34.278206 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba527a-8b48-4cef-a452-1f7f657c2c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:34.278486 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba527a-8b48-4cef-a452-1f7f657c2c9e None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:34.279219 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cba527a-8b48-4cef-a452-1f7f657c2c9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:34.279552 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cba527a-8b48-4cef-a452-1f7f657c2c9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:34.280946 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cba527a-8b48-4cef-a452-1f7f657c2c9e None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:34.299418 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 514/1035] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:25:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:34.305670 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cba527a-8b48-4cef-a452-1f7f657c2c9e None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:34.341037 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:34.397620 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e45dded-03d2-4ff3-935c-82e86e1cb711 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:34.420488 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cba527a-8b48-4cef-a452-1f7f657c2c9e None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:34.430660 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:34.432993 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6804a744-04aa-40ef-a6d7-a1a49f121082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:34.433656 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6804a744-04aa-40ef-a6d7-a1a49f121082 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:34.433938 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6804a744-04aa-40ef-a6d7-a1a49f121082 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:34.434213 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6804a744-04aa-40ef-a6d7-a1a49f121082 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:34.435033 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6804a744-04aa-40ef-a6d7-a1a49f121082 None admin] RBAC: Adding query filter params (interface=public, service_id=873b6a6b19024f50aadcd0b9d6f90225, region_id=RegionOne) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:34.435342 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6804a744-04aa-40ef-a6d7-a1a49f121082 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:34.436762 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6804a744-04aa-40ef-a6d7-a1a49f121082 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:34.442298 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 522/1036] 10.222.0.26 () {56 vars in 1251 bytes} [Mon Jul 18 16:25:34 2022] GET /identity/v3/endpoints?service_id=873b6a6b19024f50aadcd0b9d6f90225&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:34.449111 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6804a744-04aa-40ef-a6d7-a1a49f121082 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:34.461488 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e11b8fd-41fc-4da8-b364-7758646dd3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:34.462580 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e11b8fd-41fc-4da8-b364-7758646dd3a6 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:34.462867 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e11b8fd-41fc-4da8-b364-7758646dd3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:34.463146 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e11b8fd-41fc-4da8-b364-7758646dd3a6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:34.471758 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e11b8fd-41fc-4da8-b364-7758646dd3a6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:34.473231 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e11b8fd-41fc-4da8-b364-7758646dd3a6 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:34.488823 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-4e11b8fd-41fc-4da8-b364-7758646dd3a6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:25:34.488823 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:34.488823 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:34.488823 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:34.488823 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:34.488823 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:34.488823 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:34.488823 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:34.488823 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:34.488823 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:34.488823 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:34.488823 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:34.488823 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:25:34.488823 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:25:34.488823 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:25:34.488823 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:25:34.490431 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:34.490431 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:34.490431 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:25:34.490431 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:34.490431 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:25:34.490431 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:34.490431 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:34.490431 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:34.490431 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:34.490431 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:34.490431 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:34.490431 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:34.490431 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:34.490431 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:34.490431 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:34.490431 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:34.490431 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:34.490431 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:34.491376 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:34.491376 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:34.491376 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:25:34.491376 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:25:34.491376 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:25:34.491376 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:25:34.491376 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:25:34.491376 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:25:34.491376 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:25:34.491376 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:34.495430 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 515/1037] 10.222.0.26 () {56 vars in 1093 bytes} [Mon Jul 18 16:25:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:25:34.496348 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e11b8fd-41fc-4da8-b364-7758646dd3a6 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:34.560483 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6804a744-04aa-40ef-a6d7-a1a49f121082 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:34.574547 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:34.597418 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c1637b-6b97-4e31-9c9c-01d4db3ffdf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:34.598098 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c1637b-6b97-4e31-9c9c-01d4db3ffdf8 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:34.598378 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c1637b-6b97-4e31-9c9c-01d4db3ffdf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:34.598658 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c1637b-6b97-4e31-9c9c-01d4db3ffdf8 None admin] PATH_INFO: `/v3/services` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:34.600442 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c1637b-6b97-4e31-9c9c-01d4db3ffdf8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:34.601945 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c1637b-6b97-4e31-9c9c-01d4db3ffdf8 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:34.605423 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e11b8fd-41fc-4da8-b364-7758646dd3a6 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:34.609899 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 523/1038] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Jul 18 16:25:34 2022] GET /identity/v3/services => generated 1957 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:25:34.656824 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b15d1c6-16b3-4a3d-a8f5-322179fdd87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:34.657829 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b15d1c6-16b3-4a3d-a8f5-322179fdd87e None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:34.658269 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b15d1c6-16b3-4a3d-a8f5-322179fdd87e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:34.658749 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b15d1c6-16b3-4a3d-a8f5-322179fdd87e None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:34.659822 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b15d1c6-16b3-4a3d-a8f5-322179fdd87e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:34.660289 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b15d1c6-16b3-4a3d-a8f5-322179fdd87e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:34.662594 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b15d1c6-16b3-4a3d-a8f5-322179fdd87e None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:34.687538 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 516/1039] 10.222.0.26 () {56 vars in 1133 bytes} [Mon Jul 18 16:25:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:34.693504 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c1637b-6b97-4e31-9c9c-01d4db3ffdf8 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:34.810108 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c1637b-6b97-4e31-9c9c-01d4db3ffdf8 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:34.851054 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d70ff7-d9ec-4a64-9d58-ff26adeb7761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:34.851752 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d70ff7-d9ec-4a64-9d58-ff26adeb7761 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:34.852040 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d70ff7-d9ec-4a64-9d58-ff26adeb7761 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:34.852319 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d70ff7-d9ec-4a64-9d58-ff26adeb7761 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:34.899602 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:34.922975 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d70ff7-d9ec-4a64-9d58-ff26adeb7761 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:34.924481 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d70ff7-d9ec-4a64-9d58-ff26adeb7761 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:35.032978 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 524/1040] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:25:34 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 16:25:35.034053 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b15d1c6-16b3-4a3d-a8f5-322179fdd87e None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:35.071763 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:35.156350 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b15d1c6-16b3-4a3d-a8f5-322179fdd87e None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:35.191603 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4dded4-5130-466f-b579-1367f4ec4414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:35.192394 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4dded4-5130-466f-b579-1367f4ec4414 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:35.192674 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4dded4-5130-466f-b579-1367f4ec4414 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:35.192985 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4dded4-5130-466f-b579-1367f4ec4414 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:35.204033 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4dded4-5130-466f-b579-1367f4ec4414 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:35.205482 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4dded4-5130-466f-b579-1367f4ec4414 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:35.225219 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-bf4dded4-5130-466f-b579-1367f4ec4414 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:35.225219 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:35.225219 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:35.225219 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:35.225219 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:35.225219 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:35.225219 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:35.225219 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:35.225219 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:35.225219 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:35.225219 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:35.225219 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:35.225219 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:25:35.225219 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:25:35.225219 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:25:35.226179 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:25:35.226179 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:35.226179 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:35.226179 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:35.226179 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:35.226179 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:35.226179 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:35.226179 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:35.226179 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:35.226179 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:35.226179 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:35.226179 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:35.226179 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:35.226179 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:35.226179 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:35.226179 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:35.226179 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:35.227088 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:25:35.227088 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:25:35.227088 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:35.227088 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:35.227325 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 517/1041] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:35.232478 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d70ff7-d9ec-4a64-9d58-ff26adeb7761 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:35.312488 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:35.336844 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d70ff7-d9ec-4a64-9d58-ff26adeb7761 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:35.373818 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3df823-c90c-4ceb-b03a-5c5be7858f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:35.374496 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3df823-c90c-4ceb-b03a-5c5be7858f6a None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:35.374782 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3df823-c90c-4ceb-b03a-5c5be7858f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:35.375062 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3df823-c90c-4ceb-b03a-5c5be7858f6a None admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:35.375818 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3df823-c90c-4ceb-b03a-5c5be7858f6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:35.376129 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3df823-c90c-4ceb-b03a-5c5be7858f6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:35.377496 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3df823-c90c-4ceb-b03a-5c5be7858f6a None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:35.395028 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 525/1042] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:25:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:35.396477 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4dded4-5130-466f-b579-1367f4ec4414 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:35.478105 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:35.501420 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4dded4-5130-466f-b579-1367f4ec4414 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:35.535956 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd40402a-8f0b-4e23-b2c2-76baec5a7c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:35.537261 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd40402a-8f0b-4e23-b2c2-76baec5a7c43 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:35.537737 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd40402a-8f0b-4e23-b2c2-76baec5a7c43 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:35.538149 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd40402a-8f0b-4e23-b2c2-76baec5a7c43 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:35.545160 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd40402a-8f0b-4e23-b2c2-76baec5a7c43 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:35.546591 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd40402a-8f0b-4e23-b2c2-76baec5a7c43 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:35.561146 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-bd40402a-8f0b-4e23-b2c2-76baec5a7c43 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:25:35.561146 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:35.561146 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:35.561146 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:35.561146 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:35.561146 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:35.561146 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:35.561146 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:35.561146 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:35.561146 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:35.561146 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:35.561146 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:35.561146 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:25:35.561146 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:25:35.561146 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:25:35.562093 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:25:35.562093 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:35.562093 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:35.562093 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:35.562093 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:35.562093 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:35.562093 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:35.562093 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:35.562093 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:35.562093 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:35.562093 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:35.562093 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:35.562093 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:35.562093 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:35.562093 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:35.562093 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:35.562093 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:35.567899 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:25:35.567899 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:25:35.567899 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:25:35.567899 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:25:35.567899 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:25:35.567899 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:25:35.567899 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:25:35.567899 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:35.567899 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 518/1043] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:25:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:35.570069 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3df823-c90c-4ceb-b03a-5c5be7858f6a None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:35.603646 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3f47c-b518-4c5f-af1a-e8d57517f3a0 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:35.603940 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3f47c-b518-4c5f-af1a-e8d57517f3a0 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:35.604225 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3f47c-b518-4c5f-af1a-e8d57517f3a0 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:35.605728 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 519/1044] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:25:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:35.610705 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e711349-f963-4a9b-bb5c-94937592dbe9 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:35.610994 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e711349-f963-4a9b-bb5c-94937592dbe9 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:35.611273 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e711349-f963-4a9b-bb5c-94937592dbe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:35.629481 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-9e711349-f963-4a9b-bb5c-94937592dbe9 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:35.631583 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-9e711349-f963-4a9b-bb5c-94937592dbe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:35.684209 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3df823-c90c-4ceb-b03a-5c5be7858f6a None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:35.707630 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 520/1045] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:35 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:25:35.721573 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04b0751-ffc9-48b6-93cf-ff89cf0a5ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:35.722431 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04b0751-ffc9-48b6-93cf-ff89cf0a5ac0 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:35.722723 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04b0751-ffc9-48b6-93cf-ff89cf0a5ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:35.723002 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04b0751-ffc9-48b6-93cf-ff89cf0a5ac0 None admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:35.723852 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04b0751-ffc9-48b6-93cf-ff89cf0a5ac0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:35.724185 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04b0751-ffc9-48b6-93cf-ff89cf0a5ac0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:35.725660 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04b0751-ffc9-48b6-93cf-ff89cf0a5ac0 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:35.729257 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:35.730659 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e711349-f963-4a9b-bb5c-94937592dbe9 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:35.741318 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 526/1046] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 18 16:25:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:35.748972 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04b0751-ffc9-48b6-93cf-ff89cf0a5ac0 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:35.804357 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:35.846143 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e711349-f963-4a9b-bb5c-94937592dbe9 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:35.858185 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04b0751-ffc9-48b6-93cf-ff89cf0a5ac0 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:35.885636 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70127c7-0fb5-4e10-a5d4-4f14f57ffa8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:35.886435 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70127c7-0fb5-4e10-a5d4-4f14f57ffa8f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:35.886720 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70127c7-0fb5-4e10-a5d4-4f14f57ffa8f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:35.886994 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70127c7-0fb5-4e10-a5d4-4f14f57ffa8f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:35.895068 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3e0b3-e2fb-422d-a845-0733cc0d1936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:35.896506 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3e0b3-e2fb-422d-a845-0733cc0d1936 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:35.896997 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3e0b3-e2fb-422d-a845-0733cc0d1936 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:35.897452 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3e0b3-e2fb-422d-a845-0733cc0d1936 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:35.898110 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70127c7-0fb5-4e10-a5d4-4f14f57ffa8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:25:35.898659 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de3e0b3-e2fb-422d-a845-0733cc0d1936 None admin] RBAC: Adding query filter params (role.id=3394397b06be42fe9507041df2452f79, scope.project.id=eb9747e2299c4e0991fd7edb4462c7e5, user.id=5b868bfa44b94ca5b470166d39f20e54) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:35.899165 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de3e0b3-e2fb-422d-a845-0733cc0d1936 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:35.900796 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de3e0b3-e2fb-422d-a845-0733cc0d1936 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:35.905765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-b70127c7-0fb5-4e10-a5d4-4f14f57ffa8f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:25:35.905765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:35.905765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:35.905765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:35.905765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:35.905765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:35.905765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:35.905765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:35.905765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:35.905765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:35.905765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:35.905765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:35.905765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:25:35.905765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:25:35.905765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:25:35.905765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:25:35.906765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:25:35.906765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:25:35.906765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:25:35.906765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:25:35.906765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:35.906765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:35.906765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:35.906765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:35.906765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:35.906765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:35.906765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:35.906765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:35.906765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:35.906765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:35.906765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:35.906765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:35.920204 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:35.920204 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:35.920204 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:35.920204 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:35.920204 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:25:35.920204 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:25:35.920204 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:25:35.920204 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:25:35.920204 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:25:35.920204 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:25:35.920204 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:25:35.920204 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:35.920204 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 521/1047] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 18 16:25:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:35.920946 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 527/1048] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 18 16:25:35 2022] GET /identity/v3/role_assignments?user.id=5b868bfa44b94ca5b470166d39f20e54&scope.project.id=eb9747e2299c4e0991fd7edb4462c7e5&role.id=3394397b06be42fe9507041df2452f79 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:35.921031 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70127c7-0fb5-4e10-a5d4-4f14f57ffa8f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:36.036149 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70127c7-0fb5-4e10-a5d4-4f14f57ffa8f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:36.066853 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:36.078257 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bae5357-36d2-40c7-b743-5db2c6ec4f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:36.079089 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae5357-36d2-40c7-b743-5db2c6ec4f63 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:36.079439 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae5357-36d2-40c7-b743-5db2c6ec4f63 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:36.082666 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae5357-36d2-40c7-b743-5db2c6ec4f63 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:36.082666 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bae5357-36d2-40c7-b743-5db2c6ec4f63 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:36.082666 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bae5357-36d2-40c7-b743-5db2c6ec4f63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:36.082666 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bae5357-36d2-40c7-b743-5db2c6ec4f63 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:36.093497 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 522/1049] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:25:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:36.100292 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3e0b3-e2fb-422d-a845-0733cc0d1936 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:36.167435 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:36.212512 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3e0b3-e2fb-422d-a845-0733cc0d1936 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:36.251713 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a56710d-7a9c-4b51-aba0-62b12e901928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:36.252598 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a56710d-7a9c-4b51-aba0-62b12e901928 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:36.252882 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a56710d-7a9c-4b51-aba0-62b12e901928 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:36.253160 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a56710d-7a9c-4b51-aba0-62b12e901928 None admin] PATH_INFO: `/v3/users/9ce9376a557343eda36bdcc592da1708` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:36.268988 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a56710d-7a9c-4b51-aba0-62b12e901928 None admin] RBAC: Authorizing `identity:get_user(user_id=9ce9376a557343eda36bdcc592da1708)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:36.271623 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a56710d-7a9c-4b51-aba0-62b12e901928 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:36.273551 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 528/1050] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 18 16:25:36 2022] GET /identity/v3/users/9ce9376a557343eda36bdcc592da1708 => generated 285 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:36.280238 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bae5357-36d2-40c7-b743-5db2c6ec4f63 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:36.383783 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bae5357-36d2-40c7-b743-5db2c6ec4f63 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:36.420392 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0858ee-080f-4973-a30f-a8c920cced10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:36.421048 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0858ee-080f-4973-a30f-a8c920cced10 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:36.421335 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0858ee-080f-4973-a30f-a8c920cced10 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:36.421613 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0858ee-080f-4973-a30f-a8c920cced10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:36.477493 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:36.490692 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0858ee-080f-4973-a30f-a8c920cced10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:36.492182 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0858ee-080f-4973-a30f-a8c920cced10 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:36.592361 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 523/1051] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:25:36 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 16:25:36.597920 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a56710d-7a9c-4b51-aba0-62b12e901928 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:36.700810 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a56710d-7a9c-4b51-aba0-62b12e901928 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:36.719103 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:36.737023 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b3ec62-a3fa-485b-9792-1a4fdffff4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:36.737898 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b3ec62-a3fa-485b-9792-1a4fdffff4c1 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:36.738183 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b3ec62-a3fa-485b-9792-1a4fdffff4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:36.738460 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b3ec62-a3fa-485b-9792-1a4fdffff4c1 None admin] PATH_INFO: `/v3/projects/b3fc2e1486b94b5a8ba47791c6d564fc` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:36.739735 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b3ec62-a3fa-485b-9792-1a4fdffff4c1 None admin] RBAC: Authorizing `identity:get_project(project_id=b3fc2e1486b94b5a8ba47791c6d564fc)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:36.741096 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f81a16b-6719-42de-ac5e-eff2b0c0ecca None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:36.741235 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b3ec62-a3fa-485b-9792-1a4fdffff4c1 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:36.741423 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f81a16b-6719-42de-ac5e-eff2b0c0ecca None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:36.741766 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f81a16b-6719-42de-ac5e-eff2b0c0ecca None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:36.743218 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 524/1052] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:25:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:25:36.746611 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 529/1053] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:25:36 2022] GET /identity/v3/projects/b3fc2e1486b94b5a8ba47791c6d564fc => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:36.788208 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b3ec62-a3fa-485b-9792-1a4fdffff4c1 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:36.788351 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a044e2e-f778-488a-b2c5-793bc018b57d None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:36.788351 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a044e2e-f778-488a-b2c5-793bc018b57d None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:36.788351 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a044e2e-f778-488a-b2c5-793bc018b57d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:36.788351 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-2a044e2e-f778-488a-b2c5-793bc018b57d None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:36.788351 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-2a044e2e-f778-488a-b2c5-793bc018b57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:36.847967 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 525/1054] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:36 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:36.860847 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b3ec62-a3fa-485b-9792-1a4fdffff4c1 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:36.874117 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a044e2e-f778-488a-b2c5-793bc018b57d None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:36.895229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:36.900517 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df96d8f1-7375-4c92-a95c-d9817d917aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:36.901183 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df96d8f1-7375-4c92-a95c-d9817d917aa1 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:36.901464 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df96d8f1-7375-4c92-a95c-d9817d917aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:36.901739 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df96d8f1-7375-4c92-a95c-d9817d917aa1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:36.902580 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df96d8f1-7375-4c92-a95c-d9817d917aa1 None admin] RBAC: Adding query filter params (role.id=98eaadb3669f4b608d572c001742f559, scope.project.id=b3fc2e1486b94b5a8ba47791c6d564fc, user.id=9ce9376a557343eda36bdcc592da1708) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:36.902889 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df96d8f1-7375-4c92-a95c-d9817d917aa1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:36.904382 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df96d8f1-7375-4c92-a95c-d9817d917aa1 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:36.911597 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 530/1055] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 18 16:25:36 2022] GET /identity/v3/role_assignments?user.id=9ce9376a557343eda36bdcc592da1708&scope.project.id=b3fc2e1486b94b5a8ba47791c6d564fc&role.id=98eaadb3669f4b608d572c001742f559 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:36.994400 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a044e2e-f778-488a-b2c5-793bc018b57d None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:37.033675 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba91eb30-3c9e-4eb2-8fd1-a1f4dff37b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:37.034492 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba91eb30-3c9e-4eb2-8fd1-a1f4dff37b2c None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:37.034769 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba91eb30-3c9e-4eb2-8fd1-a1f4dff37b2c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:37.035047 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba91eb30-3c9e-4eb2-8fd1-a1f4dff37b2c None admin] PATH_INFO: `/v3/services/image` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:37.035739 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba91eb30-3c9e-4eb2-8fd1-a1f4dff37b2c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:37.037277 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba91eb30-3c9e-4eb2-8fd1-a1f4dff37b2c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:37.050193 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-ba91eb30-3c9e-4eb2-8fd1-a1f4dff37b2c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 16:25:37.050193 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:37.050193 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:25:37.050193 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:25:37.050193 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:25:37.050193 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:25:37.050193 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:25:37.050193 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:37.050193 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 16:25:37.050193 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:37.050193 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:25:37.050193 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:37.050193 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:37.050193 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:37.050193 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:37.050193 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:37.050193 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:37.051355 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:37.051355 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:37.051355 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:37.051355 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:37.051355 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:37.051355 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:37.051355 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:25:37.051355 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:25:37.051355 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:25:37.051355 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:25:37.051355 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:37.051355 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:37.051355 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:37.051355 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:37.051355 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:37.051355 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:37.056751 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:37.056751 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:37.056751 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:37.056751 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:37.056751 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:37.056751 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:37.056751 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:37.056751 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:37.056751 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:37.056751 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:37.056751 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:25:37.056751 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:25:37.056751 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 16:25:37.056751 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:37.056751 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 526/1056] 10.222.0.26 () {56 vars in 1105 bytes} [Mon Jul 18 16:25:36 2022] GET /identity/v3/services/image => generated 86 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:25:37.060432 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df96d8f1-7375-4c92-a95c-d9817d917aa1 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:37.117777 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:37.169103 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df96d8f1-7375-4c92-a95c-d9817d917aa1 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:37.202464 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3fe3e7-553c-479c-8c9c-f4bbe278e5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:37.203178 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3fe3e7-553c-479c-8c9c-f4bbe278e5f6 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:37.203567 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3fe3e7-553c-479c-8c9c-f4bbe278e5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:37.203881 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3fe3e7-553c-479c-8c9c-f4bbe278e5f6 None admin] PATH_INFO: `/v3/services` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:37.204579 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3fe3e7-553c-479c-8c9c-f4bbe278e5f6 None admin] RBAC: Adding query filter params (name=image) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:37.204880 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3fe3e7-553c-479c-8c9c-f4bbe278e5f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:37.206238 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3fe3e7-553c-479c-8c9c-f4bbe278e5f6 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:37.213063 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 531/1057] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:37 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:37.217649 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba91eb30-3c9e-4eb2-8fd1-a1f4dff37b2c None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:37.302870 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:37.320996 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba91eb30-3c9e-4eb2-8fd1-a1f4dff37b2c None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:37.354959 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c0fe1b-968c-40d5-920a-116be9c9b37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:37.355661 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c0fe1b-968c-40d5-920a-116be9c9b37e None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:37.355950 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c0fe1b-968c-40d5-920a-116be9c9b37e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:37.356229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c0fe1b-968c-40d5-920a-116be9c9b37e None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:37.356939 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c0fe1b-968c-40d5-920a-116be9c9b37e None admin] RBAC: Adding query filter params (type=image) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:37.357247 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c0fe1b-968c-40d5-920a-116be9c9b37e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:37.360052 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c0fe1b-968c-40d5-920a-116be9c9b37e None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:37.364334 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 527/1058] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:37 2022] GET /identity/v3/services?type=image => generated 350 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:37.370811 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3fe3e7-553c-479c-8c9c-f4bbe278e5f6 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:37.472444 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3fe3e7-553c-479c-8c9c-f4bbe278e5f6 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:37.485854 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:37.507014 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87848720-ba38-42ec-80e5-e9687d02b5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:37.507714 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87848720-ba38-42ec-80e5-e9687d02b5a6 None admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:37.508002 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87848720-ba38-42ec-80e5-e9687d02b5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:37.508283 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87848720-ba38-42ec-80e5-e9687d02b5a6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:37.509038 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87848720-ba38-42ec-80e5-e9687d02b5a6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:37.510409 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87848720-ba38-42ec-80e5-e9687d02b5a6 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:37.656286 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 532/1059] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Jul 18 16:25:37 2022] POST /identity/v3/endpoints => generated 327 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:25:37.999177 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac736a9-31fe-4920-8d66-8b2e4b768def None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:37.999547 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac736a9-31fe-4920-8d66-8b2e4b768def None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:37.999772 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac736a9-31fe-4920-8d66-8b2e4b768def None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:38.001165 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 528/1060] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:25:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:38.006758 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b49e1-1886-4bfc-8021-2826a6ac3fd6 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:38.007081 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b49e1-1886-4bfc-8021-2826a6ac3fd6 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:38.007330 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b49e1-1886-4bfc-8021-2826a6ac3fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:38.026207 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-319b49e1-1886-4bfc-8021-2826a6ac3fd6 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:38.027415 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-319b49e1-1886-4bfc-8021-2826a6ac3fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:38.103526 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 533/1061] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:38.126146 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac736a9-31fe-4920-8d66-8b2e4b768def None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:38.231071 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac736a9-31fe-4920-8d66-8b2e4b768def None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:38.264112 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b39c4d-548a-44ed-876e-8091b4662859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:38.264945 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b39c4d-548a-44ed-876e-8091b4662859 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:38.265233 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b39c4d-548a-44ed-876e-8091b4662859 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:38.265493 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b39c4d-548a-44ed-876e-8091b4662859 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:38.276029 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b39c4d-548a-44ed-876e-8091b4662859 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:25:38.284251 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-a3b39c4d-548a-44ed-876e-8091b4662859 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 16:25:38.284251 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:38.284251 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:38.284251 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:38.284251 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:38.284251 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:38.284251 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:38.284251 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:38.284251 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:38.284251 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:38.284251 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:38.284251 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:38.284251 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:25:38.284251 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:25:38.284251 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:25:38.284251 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:25:38.285186 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:25:38.285186 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:25:38.285186 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:25:38.285186 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:25:38.285186 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:38.285186 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:38.285186 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:38.285186 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:38.285186 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:38.285186 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:38.285186 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:38.285186 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:38.285186 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:38.285186 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:38.285186 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:38.285186 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:38.292535 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:38.292535 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:38.292535 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:38.292535 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:38.292535 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:25:38.292535 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:25:38.292535 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:25:38.292535 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:25:38.292535 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:25:38.292535 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:25:38.292535 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 16:25:38.292535 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:38.292535 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 529/1062] 10.222.0.26 () {56 vars in 1120 bytes} [Mon Jul 18 16:25:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:38.293232 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319b49e1-1886-4bfc-8021-2826a6ac3fd6 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:38.391178 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:38.398132 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319b49e1-1886-4bfc-8021-2826a6ac3fd6 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:38.433328 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f85410-c0aa-47ad-9261-c909ea04d117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:38.434010 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f85410-c0aa-47ad-9261-c909ea04d117 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:38.434288 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f85410-c0aa-47ad-9261-c909ea04d117 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:38.434577 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f85410-c0aa-47ad-9261-c909ea04d117 None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:38.435564 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f85410-c0aa-47ad-9261-c909ea04d117 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:38.436264 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f85410-c0aa-47ad-9261-c909ea04d117 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:38.438187 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f85410-c0aa-47ad-9261-c909ea04d117 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:38.447035 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 534/1063] 10.222.0.26 () {56 vars in 1115 bytes} [Mon Jul 18 16:25:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:38.452677 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b39c4d-548a-44ed-876e-8091b4662859 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:38.462822 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:38.555207 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b39c4d-548a-44ed-876e-8091b4662859 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:38.590253 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1d9d87-af60-4f52-9f4b-ef6a70edf20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:38.591048 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d9d87-af60-4f52-9f4b-ef6a70edf20e None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:38.594567 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d9d87-af60-4f52-9f4b-ef6a70edf20e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:38.594567 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d9d87-af60-4f52-9f4b-ef6a70edf20e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:38.602360 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1d9d87-af60-4f52-9f4b-ef6a70edf20e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:38.603820 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1d9d87-af60-4f52-9f4b-ef6a70edf20e None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:38.621524 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-2f1d9d87-af60-4f52-9f4b-ef6a70edf20e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:38.621524 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:38.621524 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:38.621524 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:38.621524 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:38.621524 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:38.621524 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:38.621524 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:38.621524 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:38.621524 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:38.621524 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:38.621524 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:38.621524 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:25:38.621524 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:25:38.621524 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:25:38.622459 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:25:38.622459 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:38.622459 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:38.622459 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:38.622459 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:38.622459 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:38.622459 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:38.622459 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:38.622459 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:38.622459 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:38.622459 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:38.622459 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:38.622459 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:38.622459 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:38.622459 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:38.622459 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:38.622459 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:38.623350 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:25:38.623350 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:25:38.623350 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:38.623350 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:38.627524 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 530/1064] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:38.628732 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f85410-c0aa-47ad-9261-c909ea04d117 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:38.741265 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f85410-c0aa-47ad-9261-c909ea04d117 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:38.775739 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3adbfa-125e-4397-b8b6-5a3222759023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:38.776421 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3adbfa-125e-4397-b8b6-5a3222759023 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:38.776709 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3adbfa-125e-4397-b8b6-5a3222759023 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:38.776986 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3adbfa-125e-4397-b8b6-5a3222759023 None admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:38.777709 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3adbfa-125e-4397-b8b6-5a3222759023 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:38.778016 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3adbfa-125e-4397-b8b6-5a3222759023 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:38.779388 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3adbfa-125e-4397-b8b6-5a3222759023 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:38.804165 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 535/1065] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:25:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:38.805947 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1d9d87-af60-4f52-9f4b-ef6a70edf20e None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:38.877287 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:38.920719 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1d9d87-af60-4f52-9f4b-ef6a70edf20e None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:38.926690 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd87998-d809-473a-9c85-150b8a58a224 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:38.927795 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd87998-d809-473a-9c85-150b8a58a224 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:38.928626 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd87998-d809-473a-9c85-150b8a58a224 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:38.929897 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 536/1066] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:25:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:25:38.934601 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509f59e-ddd9-4055-8650-84addb93318d None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:38.934891 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509f59e-ddd9-4055-8650-84addb93318d None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:38.935172 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509f59e-ddd9-4055-8650-84addb93318d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:38.953439 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-2509f59e-ddd9-4055-8650-84addb93318d None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:38.954618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-2509f59e-ddd9-4055-8650-84addb93318d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:38.959831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35790064-e7e3-47b7-9868-0879987ca51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:38.960628 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35790064-e7e3-47b7-9868-0879987ca51e None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:38.960909 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35790064-e7e3-47b7-9868-0879987ca51e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:38.961183 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35790064-e7e3-47b7-9868-0879987ca51e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:38.967955 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:38.970891 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35790064-e7e3-47b7-9868-0879987ca51e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:38.972818 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35790064-e7e3-47b7-9868-0879987ca51e None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:38.991665 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-35790064-e7e3-47b7-9868-0879987ca51e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:25:38.991665 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:38.991665 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:38.991665 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:38.991665 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:38.991665 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:38.991665 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:38.991665 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:38.991665 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:38.991665 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:38.991665 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:38.991665 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:38.991665 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:25:38.991665 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:25:38.991665 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:25:38.991665 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:25:38.992626 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:38.992626 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:38.992626 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:25:38.992626 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:38.992626 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:25:38.992626 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:38.992626 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:38.992626 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:38.992626 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:38.992626 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:38.992626 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:38.992626 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:38.992626 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:38.992626 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:38.992626 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:38.992626 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:38.992626 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:38.992626 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:38.993569 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:38.993569 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:38.993569 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:25:38.993569 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:25:38.993569 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:25:38.993569 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:25:38.993569 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:25:38.993569 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:25:38.993569 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:25:38.993569 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:38.994724 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 531/1067] 10.222.0.26 () {56 vars in 1093 bytes} [Mon Jul 18 16:25:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:38.999224 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35790064-e7e3-47b7-9868-0879987ca51e None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:39.027181 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 537/1068] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:25:39.049097 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2509f59e-ddd9-4055-8650-84addb93318d None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:39.107151 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35790064-e7e3-47b7-9868-0879987ca51e None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:39.126882 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:39.143662 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f8cee3-86f2-4276-8db6-861c87e9d223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:39.151632 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8cee3-86f2-4276-8db6-861c87e9d223 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:39.151632 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8cee3-86f2-4276-8db6-861c87e9d223 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:39.151632 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8cee3-86f2-4276-8db6-861c87e9d223 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:39.151632 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f8cee3-86f2-4276-8db6-861c87e9d223 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:39.151632 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f8cee3-86f2-4276-8db6-861c87e9d223 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:39.151632 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f8cee3-86f2-4276-8db6-861c87e9d223 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:39.158893 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2509f59e-ddd9-4055-8650-84addb93318d None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:39.166497 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 532/1069] 10.222.0.26 () {56 vars in 1133 bytes} [Mon Jul 18 16:25:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:39.172528 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f8cee3-86f2-4276-8db6-861c87e9d223 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:39.197921 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fb08cb-eee7-4a6f-a821-dba5b019aaa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:39.198720 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb08cb-eee7-4a6f-a821-dba5b019aaa6 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:39.199004 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb08cb-eee7-4a6f-a821-dba5b019aaa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:39.199277 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb08cb-eee7-4a6f-a821-dba5b019aaa6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:39.211338 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fb08cb-eee7-4a6f-a821-dba5b019aaa6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:25:39.221113 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-d9fb08cb-eee7-4a6f-a821-dba5b019aaa6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:25:39.221113 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:39.221113 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:39.221113 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:39.221113 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:39.221113 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:39.221113 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:39.221113 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:39.221113 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:39.221113 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:39.221113 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:39.221113 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:39.221113 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:25:39.221113 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:25:39.221113 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:25:39.221113 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:25:39.222256 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:25:39.222256 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:25:39.222256 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:25:39.222256 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:25:39.222256 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:39.222256 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:39.222256 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:39.222256 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:39.222256 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:39.222256 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:39.222256 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:39.222256 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:39.222256 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:39.222256 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:39.222256 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:39.222256 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:39.223125 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:39.223125 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:39.223125 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:39.223125 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:39.223125 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:25:39.223125 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:25:39.223125 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:25:39.223125 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:25:39.223125 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:25:39.223125 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:25:39.223125 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:25:39.223125 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:39.223125 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 538/1070] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 18 16:25:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:25:39.228905 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fb08cb-eee7-4a6f-a821-dba5b019aaa6 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:39.241733 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:39.288047 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f8cee3-86f2-4276-8db6-861c87e9d223 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:39.323873 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ac0190-b952-4763-817e-d3c4a3f7515c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:39.324658 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac0190-b952-4763-817e-d3c4a3f7515c None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:39.324941 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac0190-b952-4763-817e-d3c4a3f7515c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:39.325217 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac0190-b952-4763-817e-d3c4a3f7515c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:39.335548 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ac0190-b952-4763-817e-d3c4a3f7515c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:39.335924 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fb08cb-eee7-4a6f-a821-dba5b019aaa6 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:39.336994 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ac0190-b952-4763-817e-d3c4a3f7515c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:39.356270 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-18ac0190-b952-4763-817e-d3c4a3f7515c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:39.356270 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:39.356270 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:39.356270 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:39.356270 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:39.356270 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:39.356270 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:39.356270 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:39.356270 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:39.356270 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:39.356270 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:39.356270 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:39.356270 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:25:39.356270 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:25:39.356270 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:25:39.357288 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:25:39.357288 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:39.357288 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:39.357288 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:39.357288 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:39.357288 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:39.357288 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:39.357288 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:39.357288 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:39.357288 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:39.357288 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:39.357288 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:39.357288 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:39.357288 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:39.357288 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:39.357288 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:39.357288 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:39.358297 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:25:39.358297 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:25:39.358297 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:39.358297 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:39.358297 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 533/1071] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:25:39.363126 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ac0190-b952-4763-817e-d3c4a3f7515c None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:39.374598 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac4015a-dfc6-44e8-bcfa-35819e3167e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:39.375263 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4015a-dfc6-44e8-bcfa-35819e3167e8 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:39.375574 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4015a-dfc6-44e8-bcfa-35819e3167e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:39.375854 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4015a-dfc6-44e8-bcfa-35819e3167e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:39.376611 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac4015a-dfc6-44e8-bcfa-35819e3167e8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:39.376899 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac4015a-dfc6-44e8-bcfa-35819e3167e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:39.378264 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac4015a-dfc6-44e8-bcfa-35819e3167e8 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:39.391573 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 539/1072] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:25:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:39.392780 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac4015a-dfc6-44e8-bcfa-35819e3167e8 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:39.393551 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:39.467948 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ac0190-b952-4763-817e-d3c4a3f7515c None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:39.471585 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:39.510000 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c4781d-c44f-42c1-96e9-b067ea0a66c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:39.510247 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac4015a-dfc6-44e8-bcfa-35819e3167e8 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:39.510671 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c4781d-c44f-42c1-96e9-b067ea0a66c3 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:39.510956 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c4781d-c44f-42c1-96e9-b067ea0a66c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:39.511235 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c4781d-c44f-42c1-96e9-b067ea0a66c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:39.511991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c4781d-c44f-42c1-96e9-b067ea0a66c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:39.512299 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c4781d-c44f-42c1-96e9-b067ea0a66c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:39.513662 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c4781d-c44f-42c1-96e9-b067ea0a66c3 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:39.530635 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 534/1073] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:25:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:39.531703 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c4781d-c44f-42c1-96e9-b067ea0a66c3 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:39.555459 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ad9032-95a5-44aa-ab8a-d74fc28615cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:39.556075 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad9032-95a5-44aa-ab8a-d74fc28615cb None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:39.556360 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad9032-95a5-44aa-ab8a-d74fc28615cb None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:39.556636 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad9032-95a5-44aa-ab8a-d74fc28615cb None admin] PATH_INFO: `/v3/users/9ce9376a557343eda36bdcc592da1708` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:39.557845 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ad9032-95a5-44aa-ab8a-d74fc28615cb None admin] RBAC: Authorizing `identity:get_user(user_id=9ce9376a557343eda36bdcc592da1708)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:39.559278 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ad9032-95a5-44aa-ab8a-d74fc28615cb None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:39.565996 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 540/1074] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 18 16:25:39 2022] GET /identity/v3/users/9ce9376a557343eda36bdcc592da1708 => generated 285 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:39.566998 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ad9032-95a5-44aa-ab8a-d74fc28615cb None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:39.633014 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:39.646919 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c4781d-c44f-42c1-96e9-b067ea0a66c3 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:39.676150 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ad9032-95a5-44aa-ab8a-d74fc28615cb None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:39.686455 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b038c928-a29d-4898-9b67-42898bbc61cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:39.687242 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038c928-a29d-4898-9b67-42898bbc61cd None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:39.687543 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038c928-a29d-4898-9b67-42898bbc61cd None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:39.687923 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038c928-a29d-4898-9b67-42898bbc61cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:39.695758 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b038c928-a29d-4898-9b67-42898bbc61cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:39.697310 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b038c928-a29d-4898-9b67-42898bbc61cd None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:39.712049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-b038c928-a29d-4898-9b67-42898bbc61cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:25:39.712049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:39.712049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:39.712049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:39.712049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:39.712049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:39.712049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:39.712049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:39.712049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:39.712049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:39.712049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:39.712049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:39.712049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:25:39.712049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:25:39.712049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:25:39.713154 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:25:39.713154 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:39.713154 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:39.713154 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:39.713154 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:39.713154 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:39.713154 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:39.713154 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:39.713154 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:39.713154 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:39.713154 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:39.713154 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:39.713154 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:39.713154 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:39.713154 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:39.713154 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:39.713154 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:39.714071 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:25:39.714071 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:25:39.714071 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:25:39.714071 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:25:39.714071 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:25:39.714071 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:25:39.714071 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:25:39.714071 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:39.714071 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 535/1075] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:25:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:39.715633 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8223d11-e319-4baa-a565-fd8650bb012a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:39.716516 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8223d11-e319-4baa-a565-fd8650bb012a None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:39.716684 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf0095-01a5-447d-85b1-7cbeda469e2c None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:39.716780 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8223d11-e319-4baa-a565-fd8650bb012a None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:39.716953 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf0095-01a5-447d-85b1-7cbeda469e2c None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:39.717080 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8223d11-e319-4baa-a565-fd8650bb012a None admin] PATH_INFO: `/v3/projects/b3fc2e1486b94b5a8ba47791c6d564fc` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:39.717267 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf0095-01a5-447d-85b1-7cbeda469e2c None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:39.718269 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8223d11-e319-4baa-a565-fd8650bb012a None admin] RBAC: Authorizing `identity:get_project(project_id=b3fc2e1486b94b5a8ba47791c6d564fc)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:39.718530 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 536/1076] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:25:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:25:39.719220 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bf0095-01a5-447d-85b1-7cbeda469e2c None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:39.719730 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8223d11-e319-4baa-a565-fd8650bb012a None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:39.721521 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 541/1077] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:25:39 2022] GET /identity/v3/projects/b3fc2e1486b94b5a8ba47791c6d564fc => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:39.727430 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8223d11-e319-4baa-a565-fd8650bb012a None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:39.736446 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:39.836677 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bf0095-01a5-447d-85b1-7cbeda469e2c None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:39.839007 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8223d11-e319-4baa-a565-fd8650bb012a None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:39.870967 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e837f445-c0af-419d-b318-ecf2ccaa8433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:39.871655 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e837f445-c0af-419d-b318-ecf2ccaa8433 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:39.871938 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e837f445-c0af-419d-b318-ecf2ccaa8433 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:39.872212 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e837f445-c0af-419d-b318-ecf2ccaa8433 None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:39.872996 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e837f445-c0af-419d-b318-ecf2ccaa8433 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:39.873301 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e837f445-c0af-419d-b318-ecf2ccaa8433 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:39.873880 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5906e9d5-aa31-47ae-96da-feb447d556c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:39.874724 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e837f445-c0af-419d-b318-ecf2ccaa8433 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:39.874839 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906e9d5-aa31-47ae-96da-feb447d556c1 None admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:39.875039 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906e9d5-aa31-47ae-96da-feb447d556c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:39.875315 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906e9d5-aa31-47ae-96da-feb447d556c1 None admin] PATH_INFO: `/v3/projects/b3fc2e1486b94b5a8ba47791c6d564fc/users/9ce9376a557343eda36bdcc592da1708/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:39.877446 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5906e9d5-aa31-47ae-96da-feb447d556c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=b3fc2e1486b94b5a8ba47791c6d564fc, user_id=9ce9376a557343eda36bdcc592da1708, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:39.878873 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5906e9d5-aa31-47ae-96da-feb447d556c1 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:39.886712 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 537/1078] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 18 16:25:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:39.887898 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf9eb2-f25c-407a-8022-e970df594444 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:39.888187 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf9eb2-f25c-407a-8022-e970df594444 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:39.888465 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf9eb2-f25c-407a-8022-e970df594444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:39.907691 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-9ecf9eb2-f25c-407a-8022-e970df594444 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:39.908906 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-9ecf9eb2-f25c-407a-8022-e970df594444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:39.910884 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:39.952459 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 542/1079] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 18 16:25:39 2022] PUT /identity/v3/projects/b3fc2e1486b94b5a8ba47791c6d564fc/users/9ce9376a557343eda36bdcc592da1708/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:25:39.953271 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5906e9d5-aa31-47ae-96da-feb447d556c1 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:39.982122 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 538/1080] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:40.006342 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf9eb2-f25c-407a-8022-e970df594444 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:40.031238 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:40.073253 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5906e9d5-aa31-47ae-96da-feb447d556c1 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:40.119397 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcb30db-12ae-4328-95f7-2a22b660f65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:40.120333 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb30db-12ae-4328-95f7-2a22b660f65f None admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:40.120621 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb30db-12ae-4328-95f7-2a22b660f65f None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:40.120901 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb30db-12ae-4328-95f7-2a22b660f65f None admin] PATH_INFO: `/v3/projects/eb9747e2299c4e0991fd7edb4462c7e5/users/5b868bfa44b94ca5b470166d39f20e54/roles/3394397b06be42fe9507041df2452f79` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:40.121864 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf9eb2-f25c-407a-8022-e970df594444 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:40.123938 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcb30db-12ae-4328-95f7-2a22b660f65f None admin] RBAC: Authorizing `identity:create_grant(project_id=eb9747e2299c4e0991fd7edb4462c7e5, user_id=5b868bfa44b94ca5b470166d39f20e54, role_id=3394397b06be42fe9507041df2452f79)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:40.125400 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcb30db-12ae-4328-95f7-2a22b660f65f None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:40.160753 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbaa07e-e3e6-426e-a10c-f39a416fd970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:40.161521 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbaa07e-e3e6-426e-a10c-f39a416fd970 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:40.162027 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbaa07e-e3e6-426e-a10c-f39a416fd970 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:40.162329 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbaa07e-e3e6-426e-a10c-f39a416fd970 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:40.171091 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:40.184944 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 543/1081] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 18 16:25:39 2022] PUT /identity/v3/projects/eb9747e2299c4e0991fd7edb4462c7e5/users/5b868bfa44b94ca5b470166d39f20e54/roles/3394397b06be42fe9507041df2452f79 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:25:40.241612 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbaa07e-e3e6-426e-a10c-f39a416fd970 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:40.243206 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbaa07e-e3e6-426e-a10c-f39a416fd970 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:40.358674 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 539/1082] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:25:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 18 16:25:40.364760 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcb30db-12ae-4328-95f7-2a22b660f65f None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:40.473102 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcb30db-12ae-4328-95f7-2a22b660f65f None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:40.499310 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:40.508480 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9752c531-0273-46a2-ad6a-314b3f0cb1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:40.509287 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9752c531-0273-46a2-ad6a-314b3f0cb1f4 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:40.509569 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9752c531-0273-46a2-ad6a-314b3f0cb1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:40.509849 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9752c531-0273-46a2-ad6a-314b3f0cb1f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:40.521025 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9752c531-0273-46a2-ad6a-314b3f0cb1f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:40.522483 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9752c531-0273-46a2-ad6a-314b3f0cb1f4 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:40.540312 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-9752c531-0273-46a2-ad6a-314b3f0cb1f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:40.540312 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:40.540312 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:40.540312 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:40.540312 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:40.540312 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:40.540312 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:40.540312 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:40.540312 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:40.540312 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:40.540312 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:40.540312 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:40.540312 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:25:40.540312 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:25:40.540312 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:25:40.541466 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:25:40.541466 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:40.541466 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:40.541466 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:40.541466 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:40.541466 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:40.541466 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:40.541466 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:40.541466 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:40.541466 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:40.541466 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:40.541466 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:40.541466 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:40.541466 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:40.541466 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:40.541466 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:40.541466 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:40.543298 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:25:40.543298 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:25:40.543298 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:40.543298 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:40.550067 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 544/1083] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:40.554355 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbaa07e-e3e6-426e-a10c-f39a416fd970 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:40.589276 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:40.653360 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbaa07e-e3e6-426e-a10c-f39a416fd970 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:40.691267 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472becea-0d55-4073-b510-114988081526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:40.691979 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472becea-0d55-4073-b510-114988081526 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:40.692275 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472becea-0d55-4073-b510-114988081526 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:40.692553 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472becea-0d55-4073-b510-114988081526 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:40.693370 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472becea-0d55-4073-b510-114988081526 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:40.693619 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472becea-0d55-4073-b510-114988081526 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:40.695058 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472becea-0d55-4073-b510-114988081526 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:40.715778 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 540/1084] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:25:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:41.981142 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1d397-0625-45ae-8da3-9664f7a44a88 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:41.981719 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1d397-0625-45ae-8da3-9664f7a44a88 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:41.981823 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1d397-0625-45ae-8da3-9664f7a44a88 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:41.983191 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 545/1085] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:25:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:41.988463 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671de70c-e5bb-4cde-9e32-cf6f9af3ed5f None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:41.988911 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671de70c-e5bb-4cde-9e32-cf6f9af3ed5f None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:41.989953 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671de70c-e5bb-4cde-9e32-cf6f9af3ed5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:42.007016 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-671de70c-e5bb-4cde-9e32-cf6f9af3ed5f None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:42.008210 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-671de70c-e5bb-4cde-9e32-cf6f9af3ed5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:42.073604 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 541/1086] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:42.097569 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af1d397-0625-45ae-8da3-9664f7a44a88 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:42.198136 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af1d397-0625-45ae-8da3-9664f7a44a88 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:42.226759 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff3a710-e3bc-4d6a-b393-fb6861e951a4 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:42.227147 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff3a710-e3bc-4d6a-b393-fb6861e951a4 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:42.227521 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff3a710-e3bc-4d6a-b393-fb6861e951a4 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:42.228891 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 542/1087] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:25:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:25:42.231479 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e25d42-1192-4342-80aa-a2e53ebd4d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:42.232298 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e25d42-1192-4342-80aa-a2e53ebd4d41 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:42.232596 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e25d42-1192-4342-80aa-a2e53ebd4d41 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:42.232871 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e25d42-1192-4342-80aa-a2e53ebd4d41 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:42.233816 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a118b-3e41-4a9d-88f2-bad2ad9bed7e None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:42.234105 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a118b-3e41-4a9d-88f2-bad2ad9bed7e None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:42.234385 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a118b-3e41-4a9d-88f2-bad2ad9bed7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:42.243740 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e25d42-1192-4342-80aa-a2e53ebd4d41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:25:42.251298 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-241a118b-3e41-4a9d-88f2-bad2ad9bed7e None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:42.251415 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-03e25d42-1192-4342-80aa-a2e53ebd4d41 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:25:42.251415 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:42.251415 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:42.251415 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:42.251415 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:42.251415 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:42.251415 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:42.251415 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:42.251415 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:42.251415 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:42.251415 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:42.251415 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:42.251415 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:25:42.251415 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:25:42.251415 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:25:42.251415 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:25:42.252536 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:25:42.252536 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:25:42.252536 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:25:42.252536 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:25:42.252536 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:42.252536 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:42.252536 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:42.252536 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:42.252536 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:42.252536 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:42.252536 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:42.252536 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:42.252536 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:42.252536 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:42.252536 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:42.252536 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:42.253434 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:42.253434 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:42.253434 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:42.253434 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:42.253434 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:25:42.253434 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:25:42.253434 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:25:42.253434 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:25:42.253434 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:25:42.253434 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:25:42.253434 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:25:42.253434 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:25:42.254119 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-241a118b-3e41-4a9d-88f2-bad2ad9bed7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:42.254203 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 546/1088] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 18 16:25:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:42.258640 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e25d42-1192-4342-80aa-a2e53ebd4d41 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:42.278131 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:42.298338 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:42.319528 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 543/1089] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:25:42.343726 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241a118b-3e41-4a9d-88f2-bad2ad9bed7e None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:42.357326 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e25d42-1192-4342-80aa-a2e53ebd4d41 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:42.391245 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d40b7f-c5ae-4992-b660-e2bd974d9fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:42.391928 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d40b7f-c5ae-4992-b660-e2bd974d9fcd None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:42.392218 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d40b7f-c5ae-4992-b660-e2bd974d9fcd None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:42.392490 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d40b7f-c5ae-4992-b660-e2bd974d9fcd None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:42.393231 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d40b7f-c5ae-4992-b660-e2bd974d9fcd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:42.393534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d40b7f-c5ae-4992-b660-e2bd974d9fcd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:42.394934 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d40b7f-c5ae-4992-b660-e2bd974d9fcd None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:42.403318 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 547/1090] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:25:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:42.408785 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d40b7f-c5ae-4992-b660-e2bd974d9fcd None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:42.445525 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241a118b-3e41-4a9d-88f2-bad2ad9bed7e None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:42.479991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2971ca46-cb65-425a-a65d-5cc9e87187d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:42.480802 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2971ca46-cb65-425a-a65d-5cc9e87187d3 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:42.481082 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2971ca46-cb65-425a-a65d-5cc9e87187d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:42.481356 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2971ca46-cb65-425a-a65d-5cc9e87187d3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:42.492069 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2971ca46-cb65-425a-a65d-5cc9e87187d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:25:42.500483 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-2971ca46-cb65-425a-a65d-5cc9e87187d3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 16:25:42.500483 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:42.500483 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:42.500483 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:42.500483 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:42.500483 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:42.500483 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:42.500483 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:42.500483 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:42.500483 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:42.500483 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:42.500483 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:42.500483 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:25:42.500483 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:25:42.500483 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:25:42.500483 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:25:42.501428 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:25:42.501428 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:25:42.501428 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:25:42.501428 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:25:42.501428 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:42.501428 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:42.501428 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:42.501428 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:42.501428 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:42.501428 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:42.501428 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:42.501428 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:42.501428 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:42.501428 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:42.501428 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:42.501428 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:42.502277 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:42.502277 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:42.502277 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:42.502277 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:42.502277 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:25:42.502277 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:25:42.502277 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:25:42.502277 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:25:42.502277 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:25:42.502277 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:25:42.502277 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 16:25:42.502277 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:42.502277 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 544/1091] 10.222.0.26 () {56 vars in 1120 bytes} [Mon Jul 18 16:25:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:25:42.507568 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2971ca46-cb65-425a-a65d-5cc9e87187d3 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:42.508930 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:42.512538 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d40b7f-c5ae-4992-b660-e2bd974d9fcd None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:42.526605 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:42.547196 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5021f57-55c9-4c64-9f88-467c05b197a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:42.548092 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5021f57-55c9-4c64-9f88-467c05b197a4 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:42.548466 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5021f57-55c9-4c64-9f88-467c05b197a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:42.548971 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5021f57-55c9-4c64-9f88-467c05b197a4 None admin] PATH_INFO: `/v3/users/9ce9376a557343eda36bdcc592da1708` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:42.550548 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5021f57-55c9-4c64-9f88-467c05b197a4 None admin] RBAC: Authorizing `identity:get_user(user_id=9ce9376a557343eda36bdcc592da1708)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:42.552057 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5021f57-55c9-4c64-9f88-467c05b197a4 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:42.553763 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 548/1092] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 18 16:25:42 2022] GET /identity/v3/users/9ce9376a557343eda36bdcc592da1708 => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:42.559392 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5021f57-55c9-4c64-9f88-467c05b197a4 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:42.606789 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2971ca46-cb65-425a-a65d-5cc9e87187d3 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:42.641693 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878ad4cf-d54d-4d6d-9d42-92eeb78c53f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:42.642285 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ad4cf-d54d-4d6d-9d42-92eeb78c53f2 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:42.642568 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ad4cf-d54d-4d6d-9d42-92eeb78c53f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:42.642841 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ad4cf-d54d-4d6d-9d42-92eeb78c53f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:42.643609 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878ad4cf-d54d-4d6d-9d42-92eeb78c53f2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:42.643915 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878ad4cf-d54d-4d6d-9d42-92eeb78c53f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:42.645828 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878ad4cf-d54d-4d6d-9d42-92eeb78c53f2 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:42.653752 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 545/1093] 10.222.0.26 () {56 vars in 1115 bytes} [Mon Jul 18 16:25:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:42.657956 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5021f57-55c9-4c64-9f88-467c05b197a4 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:42.659472 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878ad4cf-d54d-4d6d-9d42-92eeb78c53f2 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:42.693466 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f77dd5d-e1c5-43ec-8b27-2a5e4ffeefa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:42.694124 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f77dd5d-e1c5-43ec-8b27-2a5e4ffeefa9 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:42.694408 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f77dd5d-e1c5-43ec-8b27-2a5e4ffeefa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:42.694718 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f77dd5d-e1c5-43ec-8b27-2a5e4ffeefa9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:42.730827 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:42.742844 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:42.761528 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878ad4cf-d54d-4d6d-9d42-92eeb78c53f2 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:42.763280 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f77dd5d-e1c5-43ec-8b27-2a5e4ffeefa9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:42.764940 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f77dd5d-e1c5-43ec-8b27-2a5e4ffeefa9 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:42.796590 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f524b-78c6-4daf-9ea8-23c27464ffcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:42.797394 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f524b-78c6-4daf-9ea8-23c27464ffcc None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:42.797672 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f524b-78c6-4daf-9ea8-23c27464ffcc None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:42.797940 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f524b-78c6-4daf-9ea8-23c27464ffcc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:42.808452 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445f524b-78c6-4daf-9ea8-23c27464ffcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:42.809896 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445f524b-78c6-4daf-9ea8-23c27464ffcc None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:42.827258 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-445f524b-78c6-4daf-9ea8-23c27464ffcc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:42.827258 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:42.827258 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:42.827258 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:42.827258 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:42.827258 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:42.827258 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:42.827258 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:42.827258 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:42.827258 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:42.827258 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:42.827258 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:42.827258 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:25:42.827258 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:25:42.827258 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:25:42.828210 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:25:42.828210 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:42.828210 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:42.828210 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:42.828210 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:42.828210 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:42.828210 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:42.828210 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:42.828210 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:42.828210 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:42.828210 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:42.828210 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:42.828210 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:42.828210 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:42.828210 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:42.828210 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:42.828210 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:42.829207 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:25:42.829207 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:25:42.829207 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:42.829207 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:42.829207 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 546/1094] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:25:42.835122 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f524b-78c6-4daf-9ea8-23c27464ffcc None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:42.869010 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 549/1095] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:25:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 18 16:25:42.874770 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f77dd5d-e1c5-43ec-8b27-2a5e4ffeefa9 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:42.936988 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f524b-78c6-4daf-9ea8-23c27464ffcc None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:42.939828 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:42.967146 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:42.972562 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6335a4a5-7dd1-4060-acf0-a842657b859d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:42.973238 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6335a4a5-7dd1-4060-acf0-a842657b859d None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:42.973574 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6335a4a5-7dd1-4060-acf0-a842657b859d None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:42.973843 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6335a4a5-7dd1-4060-acf0-a842657b859d None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:42.974563 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6335a4a5-7dd1-4060-acf0-a842657b859d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:42.974872 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6335a4a5-7dd1-4060-acf0-a842657b859d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:42.976315 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6335a4a5-7dd1-4060-acf0-a842657b859d None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:42.978736 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f77dd5d-e1c5-43ec-8b27-2a5e4ffeefa9 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:42.987639 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 547/1096] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:25:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:42.993576 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6335a4a5-7dd1-4060-acf0-a842657b859d None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:43.012519 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7520a7-f46b-48b9-88fc-ff99af36e2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:43.013385 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7520a7-f46b-48b9-88fc-ff99af36e2c4 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:43.013665 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7520a7-f46b-48b9-88fc-ff99af36e2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:43.013937 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7520a7-f46b-48b9-88fc-ff99af36e2c4 None admin] PATH_INFO: `/v3/projects/b3fc2e1486b94b5a8ba47791c6d564fc` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:43.015108 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7520a7-f46b-48b9-88fc-ff99af36e2c4 None admin] RBAC: Authorizing `identity:get_project(project_id=b3fc2e1486b94b5a8ba47791c6d564fc)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:43.016571 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7520a7-f46b-48b9-88fc-ff99af36e2c4 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:43.018563 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 550/1097] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:25:42 2022] GET /identity/v3/projects/b3fc2e1486b94b5a8ba47791c6d564fc => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:43.025018 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7520a7-f46b-48b9-88fc-ff99af36e2c4 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:43.093639 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6335a4a5-7dd1-4060-acf0-a842657b859d None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:43.124883 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7520a7-f46b-48b9-88fc-ff99af36e2c4 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:43.127264 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f011e581-6ccc-47c0-8c43-b2f23d816dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:43.128080 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011e581-6ccc-47c0-8c43-b2f23d816dbb None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:43.128365 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011e581-6ccc-47c0-8c43-b2f23d816dbb None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:43.128643 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011e581-6ccc-47c0-8c43-b2f23d816dbb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:43.136123 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f011e581-6ccc-47c0-8c43-b2f23d816dbb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:43.137553 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f011e581-6ccc-47c0-8c43-b2f23d816dbb None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:43.152307 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-f011e581-6ccc-47c0-8c43-b2f23d816dbb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:25:43.152307 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:43.152307 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:43.152307 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:43.152307 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:43.152307 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:43.152307 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:43.152307 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:43.152307 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:43.152307 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:43.152307 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:43.152307 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:43.152307 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:25:43.152307 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:25:43.152307 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:25:43.152307 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:25:43.153293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:43.153293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:43.153293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:25:43.153293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:43.153293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:25:43.153293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:25:43.153293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:43.153293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:43.153293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:43.153293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:43.153293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:43.153293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:43.153293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:43.153293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:43.153293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:43.153293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:43.153293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:43.153293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:43.154270 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:43.154270 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:43.154270 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:25:43.154270 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:25:43.154270 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:25:43.154270 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:25:43.154270 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:25:43.154270 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:25:43.154270 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:25:43.154270 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:43.154270 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 548/1098] 10.222.0.26 () {56 vars in 1093 bytes} [Mon Jul 18 16:25:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:25:43.159104 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f011e581-6ccc-47c0-8c43-b2f23d816dbb None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:43.161274 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ff4aa-bd5d-42b4-ae36-b47d54fbf608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:43.161953 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ff4aa-bd5d-42b4-ae36-b47d54fbf608 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:43.162332 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ff4aa-bd5d-42b4-ae36-b47d54fbf608 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:43.162610 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ff4aa-bd5d-42b4-ae36-b47d54fbf608 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:43.163445 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6ff4aa-bd5d-42b4-ae36-b47d54fbf608 None admin] RBAC: Adding query filter params (role.id=98eaadb3669f4b608d572c001742f559, scope.project.id=b3fc2e1486b94b5a8ba47791c6d564fc, user.id=9ce9376a557343eda36bdcc592da1708) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:43.163783 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6ff4aa-bd5d-42b4-ae36-b47d54fbf608 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:43.165210 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6ff4aa-bd5d-42b4-ae36-b47d54fbf608 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:43.168398 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:43.171768 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 551/1099] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 18 16:25:43 2022] GET /identity/v3/role_assignments?user.id=9ce9376a557343eda36bdcc592da1708&scope.project.id=b3fc2e1486b94b5a8ba47791c6d564fc&role.id=98eaadb3669f4b608d572c001742f559 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:43.222317 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:43.273542 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f011e581-6ccc-47c0-8c43-b2f23d816dbb None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:43.309102 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19885aec-8630-405c-8638-fee2ad6caa26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:43.309773 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19885aec-8630-405c-8638-fee2ad6caa26 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:43.310054 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19885aec-8630-405c-8638-fee2ad6caa26 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:43.310354 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19885aec-8630-405c-8638-fee2ad6caa26 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:43.311185 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19885aec-8630-405c-8638-fee2ad6caa26 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:43.311516 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19885aec-8630-405c-8638-fee2ad6caa26 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:43.312958 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19885aec-8630-405c-8638-fee2ad6caa26 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:43.331125 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 549/1100] 10.222.0.26 () {56 vars in 1133 bytes} [Mon Jul 18 16:25:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:25:43.337659 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ff4aa-bd5d-42b4-ae36-b47d54fbf608 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:43.445511 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ff4aa-bd5d-42b4-ae36-b47d54fbf608 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:43.479754 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2674fd1-8014-4066-9932-ce329878e50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:43.480315 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2674fd1-8014-4066-9932-ce329878e50e None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:43.480597 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2674fd1-8014-4066-9932-ce329878e50e None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:43.480871 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2674fd1-8014-4066-9932-ce329878e50e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:43.553381 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:43.555864 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2674fd1-8014-4066-9932-ce329878e50e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:43.557339 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2674fd1-8014-4066-9932-ce329878e50e None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:43.655350 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 552/1101] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:25:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 16:25:43.661673 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19885aec-8630-405c-8638-fee2ad6caa26 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:43.761203 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19885aec-8630-405c-8638-fee2ad6caa26 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:43.771863 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:43.796459 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f8ceb3-bc96-4e11-a177-e1614dfcb8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:43.797977 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f8ceb3-bc96-4e11-a177-e1614dfcb8fe None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:43.798503 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f8ceb3-bc96-4e11-a177-e1614dfcb8fe None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:43.799175 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f8ceb3-bc96-4e11-a177-e1614dfcb8fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:43.812083 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f8ceb3-bc96-4e11-a177-e1614dfcb8fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:43.813540 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f8ceb3-bc96-4e11-a177-e1614dfcb8fe None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:43.831653 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-c8f8ceb3-bc96-4e11-a177-e1614dfcb8fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:43.831653 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:43.831653 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:43.831653 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:43.831653 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:43.831653 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:43.831653 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:43.831653 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:43.831653 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:43.831653 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:43.831653 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:43.831653 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:43.831653 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:25:43.831653 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:25:43.831653 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:25:43.837721 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:25:43.837721 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:43.837721 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:43.837721 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:43.837721 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:43.837721 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:43.837721 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:43.837721 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:43.837721 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:43.837721 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:43.837721 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:43.837721 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:43.837721 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:43.837721 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:43.837721 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:43.837721 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:43.837721 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:43.838637 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:25:43.838637 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:25:43.838637 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:25:43.838637 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:43.838637 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 550/1102] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:43.838941 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2674fd1-8014-4066-9932-ce329878e50e None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:43.936807 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:43.939743 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2674fd1-8014-4066-9932-ce329878e50e None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:43.974312 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8fd865-fbd0-4170-a7fa-651fb0b7790e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:43.974971 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8fd865-fbd0-4170-a7fa-651fb0b7790e None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:43.975258 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8fd865-fbd0-4170-a7fa-651fb0b7790e None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:43.975561 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8fd865-fbd0-4170-a7fa-651fb0b7790e None admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:43.976279 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8fd865-fbd0-4170-a7fa-651fb0b7790e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:43.976585 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8fd865-fbd0-4170-a7fa-651fb0b7790e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:43.977946 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8fd865-fbd0-4170-a7fa-651fb0b7790e None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:43.990251 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 553/1103] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:25:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:43.996008 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f8ceb3-bc96-4e11-a177-e1614dfcb8fe None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:44.093963 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f8ceb3-bc96-4e11-a177-e1614dfcb8fe None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:44.130210 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c57344-b31e-41fc-8539-c39aa2d30129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:44.130987 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c57344-b31e-41fc-8539-c39aa2d30129 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:44.131268 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c57344-b31e-41fc-8539-c39aa2d30129 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:44.131576 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c57344-b31e-41fc-8539-c39aa2d30129 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:44.137959 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c57344-b31e-41fc-8539-c39aa2d30129 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:44.139379 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c57344-b31e-41fc-8539-c39aa2d30129 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:44.153605 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-47c57344-b31e-41fc-8539-c39aa2d30129 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:25:44.153605 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:25:44.153605 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:25:44.153605 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:25:44.153605 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:25:44.153605 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:25:44.153605 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:25:44.153605 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:25:44.153605 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:25:44.153605 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:25:44.153605 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:25:44.153605 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:25:44.153605 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:25:44.153605 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:25:44.153605 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:25:44.154688 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:25:44.154688 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:25:44.154688 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:25:44.154688 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:25:44.154688 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:25:44.154688 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:25:44.154688 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:25:44.154688 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:25:44.154688 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:25:44.154688 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:25:44.154688 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:25:44.154688 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:25:44.154688 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:25:44.154688 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:25:44.154688 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:25:44.154688 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:25:44.154688 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:25:44.155657 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:25:44.155657 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:25:44.155657 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:25:44.155657 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:25:44.155657 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:25:44.155657 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:25:44.155657 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:25:44.155657 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:25:44.155657 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 551/1104] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:25:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:25:44.161482 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8fd865-fbd0-4170-a7fa-651fb0b7790e None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:44.259837 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8fd865-fbd0-4170-a7fa-651fb0b7790e None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:44.294338 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7750952-ea79-417a-8ef7-cfdc487f7544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:44.295000 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7750952-ea79-417a-8ef7-cfdc487f7544 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:44.295279 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7750952-ea79-417a-8ef7-cfdc487f7544 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:44.295585 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7750952-ea79-417a-8ef7-cfdc487f7544 None admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:44.296358 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7750952-ea79-417a-8ef7-cfdc487f7544 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:44.296661 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7750952-ea79-417a-8ef7-cfdc487f7544 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:44.298073 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7750952-ea79-417a-8ef7-cfdc487f7544 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:44.310828 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 554/1105] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 18 16:25:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:44.317312 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c57344-b31e-41fc-8539-c39aa2d30129 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:44.343274 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:44.418025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c57344-b31e-41fc-8539-c39aa2d30129 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:44.450791 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797d4268-13f9-4817-8dfc-c0460e866bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:44.451437 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d4268-13f9-4817-8dfc-c0460e866bd0 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:44.451772 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d4268-13f9-4817-8dfc-c0460e866bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:44.452049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d4268-13f9-4817-8dfc-c0460e866bd0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:44.452874 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797d4268-13f9-4817-8dfc-c0460e866bd0 None admin] RBAC: Adding query filter params (role.id=3394397b06be42fe9507041df2452f79, scope.project.id=eb9747e2299c4e0991fd7edb4462c7e5, user.id=5b868bfa44b94ca5b470166d39f20e54) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:44.453186 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797d4268-13f9-4817-8dfc-c0460e866bd0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:44.454615 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797d4268-13f9-4817-8dfc-c0460e866bd0 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:44.461407 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 552/1106] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 18 16:25:44 2022] GET /identity/v3/role_assignments?user.id=5b868bfa44b94ca5b470166d39f20e54&scope.project.id=eb9747e2299c4e0991fd7edb4462c7e5&role.id=3394397b06be42fe9507041df2452f79 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:45.233751 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6ccc5-5b47-4c5c-9368-530f025bd4ab None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:45.234242 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6ccc5-5b47-4c5c-9368-530f025bd4ab None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:45.234242 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6ccc5-5b47-4c5c-9368-530f025bd4ab None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:45.235619 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 555/1107] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:25:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:45.241079 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b41297-e46f-47ac-8daf-cca9e9127e90 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:45.241516 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b41297-e46f-47ac-8daf-cca9e9127e90 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:45.241615 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b41297-e46f-47ac-8daf-cca9e9127e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:45.259459 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-d3b41297-e46f-47ac-8daf-cca9e9127e90 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:45.260671 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d3b41297-e46f-47ac-8daf-cca9e9127e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:45.328371 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 553/1108] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:45.353583 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c6ccc5-5b47-4c5c-9368-530f025bd4ab None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:45.374715 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:45.452257 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c6ccc5-5b47-4c5c-9368-530f025bd4ab None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:45.485939 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4c051d-d5d9-4bfc-a204-fbe8ce047572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:45.486665 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c051d-d5d9-4bfc-a204-fbe8ce047572 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:45.486948 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c051d-d5d9-4bfc-a204-fbe8ce047572 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:45.487225 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c051d-d5d9-4bfc-a204-fbe8ce047572 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:45.488405 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4c051d-d5d9-4bfc-a204-fbe8ce047572 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:45.490372 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4c051d-d5d9-4bfc-a204-fbe8ce047572 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:45.492023 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 556/1109] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:45.497859 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b41297-e46f-47ac-8daf-cca9e9127e90 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:45.515385 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:45.594578 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b41297-e46f-47ac-8daf-cca9e9127e90 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:45.628005 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bc2716-9339-4037-8de6-e7edf8ed476b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:45.628669 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc2716-9339-4037-8de6-e7edf8ed476b None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:45.628953 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc2716-9339-4037-8de6-e7edf8ed476b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:45.629323 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc2716-9339-4037-8de6-e7edf8ed476b None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:45.630717 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bc2716-9339-4037-8de6-e7edf8ed476b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:45.632200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bc2716-9339-4037-8de6-e7edf8ed476b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:45.722660 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 554/1110] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 18 16:25:45 2022] POST /identity/v3/users => generated 286 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:25:47.759602 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e957f0-3add-4746-a4e5-098e6b23a64b None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:47.760167 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e957f0-3add-4746-a4e5-098e6b23a64b None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:47.760167 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e957f0-3add-4746-a4e5-098e6b23a64b None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:47.761514 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 557/1111] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:25:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:47.766939 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75243202-0de8-42a0-a0ba-8c76607517fd None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:47.767324 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75243202-0de8-42a0-a0ba-8c76607517fd None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:47.767514 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75243202-0de8-42a0-a0ba-8c76607517fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:47.784114 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-75243202-0de8-42a0-a0ba-8c76607517fd None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:47.785317 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-75243202-0de8-42a0-a0ba-8c76607517fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:47.853296 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 555/1112] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:47.877056 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e957f0-3add-4746-a4e5-098e6b23a64b None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:47.963298 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:47.975879 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e957f0-3add-4746-a4e5-098e6b23a64b None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:48.011036 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1ab210-dfec-4598-9666-4977ab848bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:48.011850 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ab210-dfec-4598-9666-4977ab848bda None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:48.012137 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ab210-dfec-4598-9666-4977ab848bda None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:48.012416 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ab210-dfec-4598-9666-4977ab848bda None admin] PATH_INFO: `/v3/roles/729cd9c0f42d47b6a8eee01e5f5a2d28` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:48.013877 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1ab210-dfec-4598-9666-4977ab848bda None admin] RBAC: Authorizing `identity:get_role(role_id=729cd9c0f42d47b6a8eee01e5f5a2d28)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:48.015241 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1ab210-dfec-4598-9666-4977ab848bda None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:48.016644 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 558/1113] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 18 16:25:47 2022] GET /identity/v3/roles/729cd9c0f42d47b6a8eee01e5f5a2d28 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:48.022551 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75243202-0de8-42a0-a0ba-8c76607517fd None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:48.024749 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:48.118893 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75243202-0de8-42a0-a0ba-8c76607517fd None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:48.151134 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a145003a-f6b4-44a6-90e1-6a367932bf75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:48.152024 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145003a-f6b4-44a6-90e1-6a367932bf75 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:48.152307 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145003a-f6b4-44a6-90e1-6a367932bf75 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:48.152581 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145003a-f6b4-44a6-90e1-6a367932bf75 None admin] PATH_INFO: `/v3/users/27f6c06de80947a2b88d4e0741677783` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:48.166951 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a145003a-f6b4-44a6-90e1-6a367932bf75 None admin] RBAC: Authorizing `identity:get_user(user_id=27f6c06de80947a2b88d4e0741677783)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:48.168422 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a145003a-f6b4-44a6-90e1-6a367932bf75 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:48.170078 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 556/1114] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 18 16:25:48 2022] GET /identity/v3/users/27f6c06de80947a2b88d4e0741677783 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:48.178181 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1ab210-dfec-4598-9666-4977ab848bda None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:48.284856 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1ab210-dfec-4598-9666-4977ab848bda None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:48.319305 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5963c6a3-4e64-4d62-87ce-985cff716393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:48.319966 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963c6a3-4e64-4d62-87ce-985cff716393 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:48.320245 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963c6a3-4e64-4d62-87ce-985cff716393 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:48.320516 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963c6a3-4e64-4d62-87ce-985cff716393 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:48.343962 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:48.386672 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5963c6a3-4e64-4d62-87ce-985cff716393 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:48.388138 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5963c6a3-4e64-4d62-87ce-985cff716393 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:48.483930 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 559/1115] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:25:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 16:25:48.490723 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a145003a-f6b4-44a6-90e1-6a367932bf75 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:48.547807 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:48.587835 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a145003a-f6b4-44a6-90e1-6a367932bf75 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:48.621529 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a2ce9b-9d8d-469f-9bc9-bb9710b89272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:48.622377 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a2ce9b-9d8d-469f-9bc9-bb9710b89272 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:48.622661 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a2ce9b-9d8d-469f-9bc9-bb9710b89272 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:48.622969 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a2ce9b-9d8d-469f-9bc9-bb9710b89272 None admin] PATH_INFO: `/v3/projects/b3fc2e1486b94b5a8ba47791c6d564fc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:48.624155 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a2ce9b-9d8d-469f-9bc9-bb9710b89272 None admin] RBAC: Authorizing `identity:get_project(project_id=b3fc2e1486b94b5a8ba47791c6d564fc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:48.625625 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a2ce9b-9d8d-469f-9bc9-bb9710b89272 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:48.627325 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 557/1116] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:25:48 2022] GET /identity/v3/projects/b3fc2e1486b94b5a8ba47791c6d564fc => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:48.634948 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5963c6a3-4e64-4d62-87ce-985cff716393 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:48.690220 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:48.733559 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5963c6a3-4e64-4d62-87ce-985cff716393 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:48.766844 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb17a0-cce4-4193-9bdb-c038eaeb2aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:48.767516 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb17a0-cce4-4193-9bdb-c038eaeb2aca None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:48.767805 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb17a0-cce4-4193-9bdb-c038eaeb2aca None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:48.768081 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb17a0-cce4-4193-9bdb-c038eaeb2aca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:48.768911 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecb17a0-cce4-4193-9bdb-c038eaeb2aca None admin] RBAC: Adding query filter params (role.id=729cd9c0f42d47b6a8eee01e5f5a2d28, scope.project.id=b3fc2e1486b94b5a8ba47791c6d564fc, user.id=27f6c06de80947a2b88d4e0741677783) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:48.769221 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecb17a0-cce4-4193-9bdb-c038eaeb2aca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:48.770803 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecb17a0-cce4-4193-9bdb-c038eaeb2aca None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:48.776180 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 560/1117] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 18 16:25:48 2022] GET /identity/v3/role_assignments?user.id=27f6c06de80947a2b88d4e0741677783&scope.project.id=b3fc2e1486b94b5a8ba47791c6d564fc&role.id=729cd9c0f42d47b6a8eee01e5f5a2d28 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:50.879850 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7e71d-63ea-4ede-9acb-8ee852bc0e3c None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:50.880490 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7e71d-63ea-4ede-9acb-8ee852bc0e3c None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:50.880490 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7e71d-63ea-4ede-9acb-8ee852bc0e3c None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:50.881827 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 558/1118] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:25:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:50.888804 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c130ce95-10cd-4e12-b702-c7eba82d5b8a None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:50.889257 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c130ce95-10cd-4e12-b702-c7eba82d5b8a None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:50.889349 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c130ce95-10cd-4e12-b702-c7eba82d5b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:50.907912 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-c130ce95-10cd-4e12-b702-c7eba82d5b8a None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:50.909091 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-c130ce95-10cd-4e12-b702-c7eba82d5b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:50.981279 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 561/1119] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:51.006223 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d7e71d-63ea-4ede-9acb-8ee852bc0e3c None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:51.122339 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d7e71d-63ea-4ede-9acb-8ee852bc0e3c None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:51.151423 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:51.160035 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1668b8d-dcc3-4fbe-bea8-aa1fd3dd4194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:51.164305 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1668b8d-dcc3-4fbe-bea8-aa1fd3dd4194 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:51.164305 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1668b8d-dcc3-4fbe-bea8-aa1fd3dd4194 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:51.164305 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1668b8d-dcc3-4fbe-bea8-aa1fd3dd4194 None admin] PATH_INFO: `/v3/roles/729cd9c0f42d47b6a8eee01e5f5a2d28` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:51.164305 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1668b8d-dcc3-4fbe-bea8-aa1fd3dd4194 None admin] RBAC: Authorizing `identity:get_role(role_id=729cd9c0f42d47b6a8eee01e5f5a2d28)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:51.166126 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1668b8d-dcc3-4fbe-bea8-aa1fd3dd4194 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:51.167622 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 559/1120] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 18 16:25:51 2022] GET /identity/v3/roles/729cd9c0f42d47b6a8eee01e5f5a2d28 => generated 223 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:51.174685 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c130ce95-10cd-4e12-b702-c7eba82d5b8a None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:51.226661 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:51.281592 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c130ce95-10cd-4e12-b702-c7eba82d5b8a None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:51.318344 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4207297-ef10-4b09-95dc-a0db344581c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:51.319219 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4207297-ef10-4b09-95dc-a0db344581c7 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:51.319519 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4207297-ef10-4b09-95dc-a0db344581c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:51.319807 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4207297-ef10-4b09-95dc-a0db344581c7 None admin] PATH_INFO: `/v3/users/27f6c06de80947a2b88d4e0741677783` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:51.320994 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4207297-ef10-4b09-95dc-a0db344581c7 None admin] RBAC: Authorizing `identity:get_user(user_id=27f6c06de80947a2b88d4e0741677783)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:51.322424 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4207297-ef10-4b09-95dc-a0db344581c7 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:51.324118 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 562/1121] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 18 16:25:51 2022] GET /identity/v3/users/27f6c06de80947a2b88d4e0741677783 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:51.329959 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1668b8d-dcc3-4fbe-bea8-aa1fd3dd4194 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:51.427948 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1668b8d-dcc3-4fbe-bea8-aa1fd3dd4194 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:51.462254 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7393f2-65e7-4325-a639-b6b0d29a2b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:51.463107 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7393f2-65e7-4325-a639-b6b0d29a2b1c None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:51.463392 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7393f2-65e7-4325-a639-b6b0d29a2b1c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:51.463702 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7393f2-65e7-4325-a639-b6b0d29a2b1c None admin] PATH_INFO: `/v3/projects/b3fc2e1486b94b5a8ba47791c6d564fc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:51.464875 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7393f2-65e7-4325-a639-b6b0d29a2b1c None admin] RBAC: Authorizing `identity:get_project(project_id=b3fc2e1486b94b5a8ba47791c6d564fc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:51.466302 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7393f2-65e7-4325-a639-b6b0d29a2b1c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:51.468028 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 560/1122] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:25:51 2022] GET /identity/v3/projects/b3fc2e1486b94b5a8ba47791c6d564fc => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:51.474558 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4207297-ef10-4b09-95dc-a0db344581c7 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:51.573822 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4207297-ef10-4b09-95dc-a0db344581c7 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:51.590641 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:51.607315 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5320757-2510-4cef-9000-a0d6f6dd9fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:51.608247 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5320757-2510-4cef-9000-a0d6f6dd9fbc None admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:51.608521 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5320757-2510-4cef-9000-a0d6f6dd9fbc None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:51.608796 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5320757-2510-4cef-9000-a0d6f6dd9fbc None admin] PATH_INFO: `/v3/projects/b3fc2e1486b94b5a8ba47791c6d564fc/users/27f6c06de80947a2b88d4e0741677783/roles/729cd9c0f42d47b6a8eee01e5f5a2d28` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:51.611088 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5320757-2510-4cef-9000-a0d6f6dd9fbc None admin] RBAC: Authorizing `identity:create_grant(project_id=b3fc2e1486b94b5a8ba47791c6d564fc, user_id=27f6c06de80947a2b88d4e0741677783, role_id=729cd9c0f42d47b6a8eee01e5f5a2d28)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:51.612554 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5320757-2510-4cef-9000-a0d6f6dd9fbc None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:51.681819 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 563/1123] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 18 16:25:51 2022] PUT /identity/v3/projects/b3fc2e1486b94b5a8ba47791c6d564fc/users/27f6c06de80947a2b88d4e0741677783/roles/729cd9c0f42d47b6a8eee01e5f5a2d28 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:25:53.669540 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f69362-4a09-4ba7-9d43-a4564fdc3f1b None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:53.670071 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f69362-4a09-4ba7-9d43-a4564fdc3f1b None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:53.670071 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f69362-4a09-4ba7-9d43-a4564fdc3f1b None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:53.671461 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 561/1124] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:25:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:53.676742 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9495a9-693c-4267-97fd-bd0f02f742ea None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:53.677152 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9495a9-693c-4267-97fd-bd0f02f742ea None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:53.677300 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9495a9-693c-4267-97fd-bd0f02f742ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:53.694049 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-4c9495a9-693c-4267-97fd-bd0f02f742ea None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:53.695222 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-4c9495a9-693c-4267-97fd-bd0f02f742ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:53.760426 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 564/1125] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:53.783191 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f69362-4a09-4ba7-9d43-a4564fdc3f1b None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:53.849093 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:53.888623 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f69362-4a09-4ba7-9d43-a4564fdc3f1b None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:53.921904 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3ba751-0bbc-456e-83ee-0ced664e0d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:53.922694 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ba751-0bbc-456e-83ee-0ced664e0d32 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:53.922980 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ba751-0bbc-456e-83ee-0ced664e0d32 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:53.923260 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ba751-0bbc-456e-83ee-0ced664e0d32 None admin] PATH_INFO: `/v3/roles/729cd9c0f42d47b6a8eee01e5f5a2d28` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:53.924755 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3ba751-0bbc-456e-83ee-0ced664e0d32 None admin] RBAC: Authorizing `identity:get_role(role_id=729cd9c0f42d47b6a8eee01e5f5a2d28)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:53.926760 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3ba751-0bbc-456e-83ee-0ced664e0d32 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:53.928254 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 562/1126] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 18 16:25:53 2022] GET /identity/v3/roles/729cd9c0f42d47b6a8eee01e5f5a2d28 => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:53.934296 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9495a9-693c-4267-97fd-bd0f02f742ea None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:54.031908 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9495a9-693c-4267-97fd-bd0f02f742ea None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:54.041301 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:54.071147 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496853ca-490d-4a47-83f7-293aa18082ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:54.072052 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496853ca-490d-4a47-83f7-293aa18082ed None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:54.072352 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496853ca-490d-4a47-83f7-293aa18082ed None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:54.072625 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496853ca-490d-4a47-83f7-293aa18082ed None admin] PATH_INFO: `/v3/users/27f6c06de80947a2b88d4e0741677783` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:54.073801 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496853ca-490d-4a47-83f7-293aa18082ed None admin] RBAC: Authorizing `identity:get_user(user_id=27f6c06de80947a2b88d4e0741677783)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:54.075222 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496853ca-490d-4a47-83f7-293aa18082ed None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:54.076926 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 565/1127] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 18 16:25:53 2022] GET /identity/v3/users/27f6c06de80947a2b88d4e0741677783 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:54.082903 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3ba751-0bbc-456e-83ee-0ced664e0d32 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:54.181073 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3ba751-0bbc-456e-83ee-0ced664e0d32 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:54.209572 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:54.214708 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45206ded-ea62-496d-8ed2-02b0b0448d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:54.222731 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45206ded-ea62-496d-8ed2-02b0b0448d62 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:54.222731 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45206ded-ea62-496d-8ed2-02b0b0448d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:54.222731 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45206ded-ea62-496d-8ed2-02b0b0448d62 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:54.282218 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45206ded-ea62-496d-8ed2-02b0b0448d62 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:54.283695 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45206ded-ea62-496d-8ed2-02b0b0448d62 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:54.380064 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 563/1128] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:25:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 16:25:54.385995 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496853ca-490d-4a47-83f7-293aa18082ed None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:54.486954 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496853ca-490d-4a47-83f7-293aa18082ed None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:54.520702 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9ba5c2-3ae2-49bf-9b7b-ee89b8bc3084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:54.521563 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ba5c2-3ae2-49bf-9b7b-ee89b8bc3084 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:54.521842 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ba5c2-3ae2-49bf-9b7b-ee89b8bc3084 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:54.522115 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ba5c2-3ae2-49bf-9b7b-ee89b8bc3084 None admin] PATH_INFO: `/v3/projects/b3fc2e1486b94b5a8ba47791c6d564fc` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:54.523284 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9ba5c2-3ae2-49bf-9b7b-ee89b8bc3084 None admin] RBAC: Authorizing `identity:get_project(project_id=b3fc2e1486b94b5a8ba47791c6d564fc)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:54.524740 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9ba5c2-3ae2-49bf-9b7b-ee89b8bc3084 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:54.526443 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 566/1129] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:25:54 2022] GET /identity/v3/projects/b3fc2e1486b94b5a8ba47791c6d564fc => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:54.532590 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45206ded-ea62-496d-8ed2-02b0b0448d62 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:54.562711 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:54.629318 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45206ded-ea62-496d-8ed2-02b0b0448d62 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:54.666023 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0fc1bb-3be2-434f-83c0-df611b7e946a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:54.666782 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0fc1bb-3be2-434f-83c0-df611b7e946a None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:54.667269 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0fc1bb-3be2-434f-83c0-df611b7e946a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:54.667781 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0fc1bb-3be2-434f-83c0-df611b7e946a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:54.668908 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0fc1bb-3be2-434f-83c0-df611b7e946a None admin] RBAC: Adding query filter params (role.id=729cd9c0f42d47b6a8eee01e5f5a2d28, scope.project.id=b3fc2e1486b94b5a8ba47791c6d564fc, user.id=27f6c06de80947a2b88d4e0741677783) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:25:54.669222 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0fc1bb-3be2-434f-83c0-df611b7e946a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:54.670638 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0fc1bb-3be2-434f-83c0-df611b7e946a None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:54.676666 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 564/1130] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 18 16:25:54 2022] GET /identity/v3/role_assignments?user.id=27f6c06de80947a2b88d4e0741677783&scope.project.id=b3fc2e1486b94b5a8ba47791c6d564fc&role.id=729cd9c0f42d47b6a8eee01e5f5a2d28 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:56.716615 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b5ad7-260b-489a-b7e5-9d58daef72d8 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:56.717172 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b5ad7-260b-489a-b7e5-9d58daef72d8 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:56.717172 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b5ad7-260b-489a-b7e5-9d58daef72d8 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:56.728030 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 567/1131] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:25:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:56.728209 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf6471e-93fd-436c-b085-a7ccebe24193 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:56.728209 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf6471e-93fd-436c-b085-a7ccebe24193 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:56.728209 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf6471e-93fd-436c-b085-a7ccebe24193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:56.742434 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-4cf6471e-93fd-436c-b085-a7ccebe24193 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:56.743629 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-4cf6471e-93fd-436c-b085-a7ccebe24193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:56.809460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 565/1132] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:25:56.833255 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4b5ad7-260b-489a-b7e5-9d58daef72d8 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:56.879889 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:56.938608 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4b5ad7-260b-489a-b7e5-9d58daef72d8 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:56.973219 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b375d162-7bda-4309-bcc8-422da718233f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:56.973949 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b375d162-7bda-4309-bcc8-422da718233f None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:56.974234 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b375d162-7bda-4309-bcc8-422da718233f None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:56.974509 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b375d162-7bda-4309-bcc8-422da718233f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:56.975708 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b375d162-7bda-4309-bcc8-422da718233f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:56.977158 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b375d162-7bda-4309-bcc8-422da718233f None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:56.978780 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 568/1133] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:56.984711 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf6471e-93fd-436c-b085-a7ccebe24193 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:57.011926 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:57.083267 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf6471e-93fd-436c-b085-a7ccebe24193 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:57.116558 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe34fa43-50c4-40e6-9478-cf71a1427c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:57.117248 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe34fa43-50c4-40e6-9478-cf71a1427c72 None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:57.117530 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe34fa43-50c4-40e6-9478-cf71a1427c72 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:57.117804 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe34fa43-50c4-40e6-9478-cf71a1427c72 None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:57.118555 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe34fa43-50c4-40e6-9478-cf71a1427c72 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:57.120013 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe34fa43-50c4-40e6-9478-cf71a1427c72 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:57.254443 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 566/1134] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jul 18 16:25:56 2022] POST /identity/v3/projects => generated 311 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:25:59.296211 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9b94d-3571-472c-a049-2c8659eadf03 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:59.296732 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9b94d-3571-472c-a049-2c8659eadf03 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:59.296852 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9b94d-3571-472c-a049-2c8659eadf03 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:59.298234 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 569/1135] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:25:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:25:59.304202 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d04216-9129-4c26-af12-cefcb363fd9c None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:59.304661 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d04216-9129-4c26-af12-cefcb363fd9c None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:59.304797 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d04216-9129-4c26-af12-cefcb363fd9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:59.321729 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-a5d04216-9129-4c26-af12-cefcb363fd9c None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:25:59.322900 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a5d04216-9129-4c26-af12-cefcb363fd9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:25:59.389611 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 567/1136] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:25:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:25:59.414829 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae9b94d-3571-472c-a049-2c8659eadf03 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:59.516400 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae9b94d-3571-472c-a049-2c8659eadf03 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:59.529118 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:59.550409 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6a4bbd-38da-4acb-86b9-62712c2be4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:59.551155 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6a4bbd-38da-4acb-86b9-62712c2be4dd None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:59.551431 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6a4bbd-38da-4acb-86b9-62712c2be4dd None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:59.551732 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6a4bbd-38da-4acb-86b9-62712c2be4dd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:59.552905 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6a4bbd-38da-4acb-86b9-62712c2be4dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:59.554327 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6a4bbd-38da-4acb-86b9-62712c2be4dd None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:59.556043 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 570/1137] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:25:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:25:59.562049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d04216-9129-4c26-af12-cefcb363fd9c None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:25:59.573895 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:25:59.661973 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d04216-9129-4c26-af12-cefcb363fd9c None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:25:59.694879 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23377cdf-9ce0-4433-a11a-f12ebc89a4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:25:59.695554 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23377cdf-9ce0-4433-a11a-f12ebc89a4a9 None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:25:59.695837 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23377cdf-9ce0-4433-a11a-f12ebc89a4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:25:59.696109 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23377cdf-9ce0-4433-a11a-f12ebc89a4a9 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:25:59.696819 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23377cdf-9ce0-4433-a11a-f12ebc89a4a9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:25:59.698291 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23377cdf-9ce0-4433-a11a-f12ebc89a4a9 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:25:59.802735 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 568/1138] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 18 16:25:59 2022] POST /identity/v3/users => generated 286 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:26:01.856908 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72dfe6-93fb-4977-92d6-021aef260b90 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:01.857431 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72dfe6-93fb-4977-92d6-021aef260b90 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:01.857524 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72dfe6-93fb-4977-92d6-021aef260b90 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:01.858816 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 571/1139] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:26:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:26:01.864760 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9110b8-24d4-4d66-a4ab-0edc800bbd6a None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:01.865196 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9110b8-24d4-4d66-a4ab-0edc800bbd6a None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:01.865302 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9110b8-24d4-4d66-a4ab-0edc800bbd6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:01.883164 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-aa9110b8-24d4-4d66-a4ab-0edc800bbd6a None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:26:01.884367 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-aa9110b8-24d4-4d66-a4ab-0edc800bbd6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:26:01.941897 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:01.949826 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 569/1140] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:26:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:26:01.973175 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa72dfe6-93fb-4977-92d6-021aef260b90 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:02.076281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa72dfe6-93fb-4977-92d6-021aef260b90 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:02.109548 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e74f1b-c13d-44e4-ac0c-a49fe3bd3dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:02.110272 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e74f1b-c13d-44e4-ac0c-a49fe3bd3dd0 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:02.110557 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e74f1b-c13d-44e4-ac0c-a49fe3bd3dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:02.110832 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e74f1b-c13d-44e4-ac0c-a49fe3bd3dd0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:02.121317 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e74f1b-c13d-44e4-ac0c-a49fe3bd3dd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:26:02.129144 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-06e74f1b-c13d-44e4-ac0c-a49fe3bd3dd0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:26:02.129144 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:26:02.129144 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:26:02.129144 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:26:02.129144 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:26:02.129144 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:26:02.129144 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:26:02.129144 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:26:02.129144 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:26:02.129144 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:26:02.129144 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:26:02.129144 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:26:02.129144 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:26:02.129144 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:26:02.129144 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:26:02.129144 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:26:02.130241 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:26:02.130241 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:26:02.130241 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:26:02.130241 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:26:02.130241 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:26:02.130241 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:26:02.130241 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:26:02.130241 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:26:02.130241 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:26:02.130241 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:26:02.130241 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:26:02.130241 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:26:02.130241 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:26:02.130241 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:26:02.130241 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:26:02.130241 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:26:02.131380 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:26:02.131380 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:26:02.131380 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:26:02.131380 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:26:02.131380 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:26:02.131380 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:26:02.131380 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:26:02.131380 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:26:02.131380 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:26:02.131380 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:26:02.131380 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:26:02.131380 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:26:02.131380 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 572/1141] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 18 16:26:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:26:02.136479 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9110b8-24d4-4d66-a4ab-0edc800bbd6a None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:02.232326 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9110b8-24d4-4d66-a4ab-0edc800bbd6a None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:02.268053 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaad394-8e64-49ae-8a88-66f799719c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:02.268745 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaad394-8e64-49ae-8a88-66f799719c97 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:02.269024 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaad394-8e64-49ae-8a88-66f799719c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:02.269296 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaad394-8e64-49ae-8a88-66f799719c97 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:02.270036 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaad394-8e64-49ae-8a88-66f799719c97 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:26:02.270336 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaad394-8e64-49ae-8a88-66f799719c97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:02.271728 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaad394-8e64-49ae-8a88-66f799719c97 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:02.279999 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 570/1142] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:26:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:02.286719 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e74f1b-c13d-44e4-ac0c-a49fe3bd3dd0 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:02.329119 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:02.389312 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e74f1b-c13d-44e4-ac0c-a49fe3bd3dd0 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:02.423979 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a0654b-97fb-4d9c-a52e-444a7510ccfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:02.424947 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a0654b-97fb-4d9c-a52e-444a7510ccfd None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:02.425234 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a0654b-97fb-4d9c-a52e-444a7510ccfd None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:02.425515 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a0654b-97fb-4d9c-a52e-444a7510ccfd None admin] PATH_INFO: `/v3/users/c555a90289274f9989bd4e919a14c943` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:02.438524 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a0654b-97fb-4d9c-a52e-444a7510ccfd None admin] RBAC: Authorizing `identity:get_user(user_id=c555a90289274f9989bd4e919a14c943)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:02.440030 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a0654b-97fb-4d9c-a52e-444a7510ccfd None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:02.441836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 573/1143] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 18 16:26:02 2022] GET /identity/v3/users/c555a90289274f9989bd4e919a14c943 => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:02.447863 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaad394-8e64-49ae-8a88-66f799719c97 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:02.506138 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:02.549462 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaad394-8e64-49ae-8a88-66f799719c97 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:02.586638 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4bd72-5050-418e-872f-13b78a8b2a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:02.587294 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4bd72-5050-418e-872f-13b78a8b2a35 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:02.587616 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4bd72-5050-418e-872f-13b78a8b2a35 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:02.587896 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4bd72-5050-418e-872f-13b78a8b2a35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:02.667267 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d4bd72-5050-418e-872f-13b78a8b2a35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:02.668989 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d4bd72-5050-418e-872f-13b78a8b2a35 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:02.723882 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:02.767786 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 571/1144] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:26:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 16:26:02.774366 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a0654b-97fb-4d9c-a52e-444a7510ccfd None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:02.872517 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:02.876265 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a0654b-97fb-4d9c-a52e-444a7510ccfd None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:02.910141 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea91267-4684-43de-b873-85aba20ce9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:02.910993 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea91267-4684-43de-b873-85aba20ce9b1 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:02.911331 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea91267-4684-43de-b873-85aba20ce9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:02.911628 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea91267-4684-43de-b873-85aba20ce9b1 None admin] PATH_INFO: `/v3/projects/985bb364a7d24a9aa4fa40a6356071b1` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:02.912817 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea91267-4684-43de-b873-85aba20ce9b1 None admin] RBAC: Authorizing `identity:get_project(project_id=985bb364a7d24a9aa4fa40a6356071b1)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:02.914237 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea91267-4684-43de-b873-85aba20ce9b1 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:02.923404 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 574/1145] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:26:02 2022] GET /identity/v3/projects/985bb364a7d24a9aa4fa40a6356071b1 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:02.923572 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4bd72-5050-418e-872f-13b78a8b2a35 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:03.019726 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4bd72-5050-418e-872f-13b78a8b2a35 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:03.055154 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ccf412-0efe-419c-bd41-3660b96fb8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:03.055833 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ccf412-0efe-419c-bd41-3660b96fb8b5 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:03.056118 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ccf412-0efe-419c-bd41-3660b96fb8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:03.056390 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ccf412-0efe-419c-bd41-3660b96fb8b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:03.057212 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ccf412-0efe-419c-bd41-3660b96fb8b5 None admin] RBAC: Adding query filter params (role.id=98eaadb3669f4b608d572c001742f559, scope.project.id=985bb364a7d24a9aa4fa40a6356071b1, user.id=c555a90289274f9989bd4e919a14c943) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:26:03.057516 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ccf412-0efe-419c-bd41-3660b96fb8b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:03.058950 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ccf412-0efe-419c-bd41-3660b96fb8b5 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:03.064326 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 572/1146] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 18 16:26:02 2022] GET /identity/v3/role_assignments?user.id=c555a90289274f9989bd4e919a14c943&scope.project.id=985bb364a7d24a9aa4fa40a6356071b1&role.id=98eaadb3669f4b608d572c001742f559 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:05.061384 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3b185-30f8-4fcb-a6a9-2a0278f4933f None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:05.061941 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3b185-30f8-4fcb-a6a9-2a0278f4933f None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:05.061941 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3b185-30f8-4fcb-a6a9-2a0278f4933f None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:05.063310 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 575/1147] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:26:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:26:05.068774 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300a56f-4d94-495c-af50-85fadf39a70a None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:05.069171 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300a56f-4d94-495c-af50-85fadf39a70a None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:05.069334 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300a56f-4d94-495c-af50-85fadf39a70a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:05.086141 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-c300a56f-4d94-495c-af50-85fadf39a70a None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:26:05.087252 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-c300a56f-4d94-495c-af50-85fadf39a70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:26:05.101475 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:05.153406 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 573/1148] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:26:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:26:05.177448 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d3b185-30f8-4fcb-a6a9-2a0278f4933f None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:05.278236 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d3b185-30f8-4fcb-a6a9-2a0278f4933f None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:05.314785 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e055bb-19dd-487a-898e-64c491559faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:05.316433 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e055bb-19dd-487a-898e-64c491559faa None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:05.316711 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e055bb-19dd-487a-898e-64c491559faa None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:05.316986 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e055bb-19dd-487a-898e-64c491559faa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:05.327474 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e055bb-19dd-487a-898e-64c491559faa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:26:05.335638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-22e055bb-19dd-487a-898e-64c491559faa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:26:05.335638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:26:05.335638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:26:05.335638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:26:05.335638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:26:05.335638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:26:05.335638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:26:05.335638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:26:05.335638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:26:05.335638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:26:05.335638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:26:05.335638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:26:05.335638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:26:05.335638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:26:05.335638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:26:05.335638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:26:05.336815 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:26:05.336815 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:26:05.336815 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:26:05.336815 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:26:05.336815 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:26:05.336815 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:26:05.336815 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:26:05.336815 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:26:05.336815 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:26:05.336815 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:26:05.336815 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:26:05.336815 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:26:05.336815 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:26:05.336815 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:26:05.336815 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:26:05.336815 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:26:05.337875 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:26:05.337875 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:26:05.337875 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:26:05.337875 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:26:05.337875 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:26:05.337875 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:26:05.337875 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:26:05.337875 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:26:05.337875 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:26:05.337875 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:26:05.337875 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:26:05.337875 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:26:05.337875 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 576/1149] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 18 16:26:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:26:05.342978 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300a56f-4d94-495c-af50-85fadf39a70a None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:05.444346 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300a56f-4d94-495c-af50-85fadf39a70a None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:05.478072 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c23688-f8a7-42e2-9cb5-3df78242e5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:05.478741 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c23688-f8a7-42e2-9cb5-3df78242e5aa None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:05.479027 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c23688-f8a7-42e2-9cb5-3df78242e5aa None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:05.479305 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c23688-f8a7-42e2-9cb5-3df78242e5aa None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:05.480082 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c23688-f8a7-42e2-9cb5-3df78242e5aa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:26:05.480385 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c23688-f8a7-42e2-9cb5-3df78242e5aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:05.481773 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c23688-f8a7-42e2-9cb5-3df78242e5aa None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:05.492070 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 574/1150] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:26:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:05.497898 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e055bb-19dd-487a-898e-64c491559faa None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:05.527735 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:05.602651 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e055bb-19dd-487a-898e-64c491559faa None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:05.636603 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3971448-58ee-41eb-9925-4096d33d5fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:05.637470 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3971448-58ee-41eb-9925-4096d33d5fd3 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:05.637754 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3971448-58ee-41eb-9925-4096d33d5fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:05.638030 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3971448-58ee-41eb-9925-4096d33d5fd3 None admin] PATH_INFO: `/v3/users/c555a90289274f9989bd4e919a14c943` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:05.639206 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3971448-58ee-41eb-9925-4096d33d5fd3 None admin] RBAC: Authorizing `identity:get_user(user_id=c555a90289274f9989bd4e919a14c943)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:05.640677 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3971448-58ee-41eb-9925-4096d33d5fd3 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:05.642316 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 577/1151] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 18 16:26:05 2022] GET /identity/v3/users/c555a90289274f9989bd4e919a14c943 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:05.648206 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c23688-f8a7-42e2-9cb5-3df78242e5aa None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:05.663249 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:05.746877 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c23688-f8a7-42e2-9cb5-3df78242e5aa None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:05.780592 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7632d9ea-2704-49e6-a455-c5e587cac203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:05.781437 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7632d9ea-2704-49e6-a455-c5e587cac203 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:05.781719 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7632d9ea-2704-49e6-a455-c5e587cac203 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:05.781991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7632d9ea-2704-49e6-a455-c5e587cac203 None admin] PATH_INFO: `/v3/projects/985bb364a7d24a9aa4fa40a6356071b1` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:05.783179 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7632d9ea-2704-49e6-a455-c5e587cac203 None admin] RBAC: Authorizing `identity:get_project(project_id=985bb364a7d24a9aa4fa40a6356071b1)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:05.784794 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7632d9ea-2704-49e6-a455-c5e587cac203 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:05.786480 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 575/1152] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:26:05 2022] GET /identity/v3/projects/985bb364a7d24a9aa4fa40a6356071b1 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:05.792943 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3971448-58ee-41eb-9925-4096d33d5fd3 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:05.887616 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:05.895287 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3971448-58ee-41eb-9925-4096d33d5fd3 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:05.929294 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09944781-bf4c-47ad-a212-5b2b68c9a8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:05.930184 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09944781-bf4c-47ad-a212-5b2b68c9a8ae None admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:05.930468 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09944781-bf4c-47ad-a212-5b2b68c9a8ae None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:05.930744 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09944781-bf4c-47ad-a212-5b2b68c9a8ae None admin] PATH_INFO: `/v3/projects/985bb364a7d24a9aa4fa40a6356071b1/users/c555a90289274f9989bd4e919a14c943/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:05.932862 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09944781-bf4c-47ad-a212-5b2b68c9a8ae None admin] RBAC: Authorizing `identity:create_grant(project_id=985bb364a7d24a9aa4fa40a6356071b1, user_id=c555a90289274f9989bd4e919a14c943, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:05.934905 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09944781-bf4c-47ad-a212-5b2b68c9a8ae None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:06.132368 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 578/1153] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 18 16:26:05 2022] PUT /identity/v3/projects/985bb364a7d24a9aa4fa40a6356071b1/users/c555a90289274f9989bd4e919a14c943/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:26:08.142542 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd48167-d8ce-442b-afa3-7351a221394a None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:08.143721 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd48167-d8ce-442b-afa3-7351a221394a None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:08.143721 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd48167-d8ce-442b-afa3-7351a221394a None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:08.144960 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 576/1154] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:26:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:26:08.150168 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b11cc8-8c2d-4c62-a92a-e2c966f25866 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:08.150595 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b11cc8-8c2d-4c62-a92a-e2c966f25866 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:08.150704 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b11cc8-8c2d-4c62-a92a-e2c966f25866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:08.167428 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-69b11cc8-8c2d-4c62-a92a-e2c966f25866 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:26:08.168709 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-69b11cc8-8c2d-4c62-a92a-e2c966f25866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:26:08.233267 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 579/1155] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:26:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:26:08.257225 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd48167-d8ce-442b-afa3-7351a221394a None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:08.338412 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:08.357354 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd48167-d8ce-442b-afa3-7351a221394a None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:08.391910 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4ce711-1360-424a-828d-62aab77f1d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:08.392792 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ce711-1360-424a-828d-62aab77f1d83 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:08.392988 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ce711-1360-424a-828d-62aab77f1d83 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:08.393267 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ce711-1360-424a-828d-62aab77f1d83 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:08.403264 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4ce711-1360-424a-828d-62aab77f1d83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:26:08.410908 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-9d4ce711-1360-424a-828d-62aab77f1d83 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:26:08.410908 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:26:08.410908 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:26:08.410908 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:26:08.410908 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:26:08.410908 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:26:08.410908 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:26:08.410908 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:26:08.410908 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:26:08.410908 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:26:08.410908 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:26:08.410908 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:26:08.410908 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:26:08.410908 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:26:08.410908 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:26:08.410908 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:26:08.412042 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:26:08.412042 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:26:08.412042 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:26:08.412042 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:26:08.412042 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:26:08.412042 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:26:08.412042 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:26:08.412042 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:26:08.412042 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:26:08.412042 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:26:08.412042 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:26:08.412042 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:26:08.412042 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:26:08.412042 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:26:08.412042 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:26:08.412042 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:26:08.413106 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:26:08.413106 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:26:08.413106 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:26:08.413106 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:26:08.413106 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:26:08.413106 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:26:08.413106 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:26:08.413106 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:26:08.413106 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:26:08.413106 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:26:08.413106 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:26:08.413106 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:26:08.413106 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 577/1156] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 18 16:26:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:26:08.418594 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b11cc8-8c2d-4c62-a92a-e2c966f25866 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:08.500072 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:08.516071 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b11cc8-8c2d-4c62-a92a-e2c966f25866 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:08.549870 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f667b-ff60-4767-98e5-9904fb93a982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:08.550543 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f667b-ff60-4767-98e5-9904fb93a982 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:08.550831 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f667b-ff60-4767-98e5-9904fb93a982 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:08.551111 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f667b-ff60-4767-98e5-9904fb93a982 None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:08.551895 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60f667b-ff60-4767-98e5-9904fb93a982 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:26:08.552231 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60f667b-ff60-4767-98e5-9904fb93a982 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:08.553610 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60f667b-ff60-4767-98e5-9904fb93a982 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:08.561657 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 580/1157] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:26:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:08.568493 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4ce711-1360-424a-828d-62aab77f1d83 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:08.666988 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4ce711-1360-424a-828d-62aab77f1d83 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:08.700933 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2be714-823c-4711-ba6e-16ed1cfb185d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:08.701790 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2be714-823c-4711-ba6e-16ed1cfb185d None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:08.702069 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2be714-823c-4711-ba6e-16ed1cfb185d None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:08.702341 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2be714-823c-4711-ba6e-16ed1cfb185d None admin] PATH_INFO: `/v3/users/c555a90289274f9989bd4e919a14c943` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:08.703588 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2be714-823c-4711-ba6e-16ed1cfb185d None admin] RBAC: Authorizing `identity:get_user(user_id=c555a90289274f9989bd4e919a14c943)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:08.705037 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2be714-823c-4711-ba6e-16ed1cfb185d None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:08.706694 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 578/1158] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 18 16:26:08 2022] GET /identity/v3/users/c555a90289274f9989bd4e919a14c943 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:08.712568 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f667b-ff60-4767-98e5-9904fb93a982 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:08.811387 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f667b-ff60-4767-98e5-9904fb93a982 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:08.848082 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b5c890-a54b-4ed3-8273-04db4bf1c60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:08.848731 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5c890-a54b-4ed3-8273-04db4bf1c60c None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:08.849011 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5c890-a54b-4ed3-8273-04db4bf1c60c None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:08.849291 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5c890-a54b-4ed3-8273-04db4bf1c60c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:08.865518 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:08.918795 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b5c890-a54b-4ed3-8273-04db4bf1c60c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:08.920227 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b5c890-a54b-4ed3-8273-04db4bf1c60c None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:09.018490 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 581/1159] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:26:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 16:26:09.024247 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2be714-823c-4711-ba6e-16ed1cfb185d None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:09.035519 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:09.125946 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2be714-823c-4711-ba6e-16ed1cfb185d None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:09.159708 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0ac2d3-3c91-429b-ae21-2d225fab5031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:09.160572 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0ac2d3-3c91-429b-ae21-2d225fab5031 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:09.160856 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0ac2d3-3c91-429b-ae21-2d225fab5031 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:09.161133 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0ac2d3-3c91-429b-ae21-2d225fab5031 None admin] PATH_INFO: `/v3/projects/985bb364a7d24a9aa4fa40a6356071b1` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:09.162311 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0ac2d3-3c91-429b-ae21-2d225fab5031 None admin] RBAC: Authorizing `identity:get_project(project_id=985bb364a7d24a9aa4fa40a6356071b1)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:09.163771 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0ac2d3-3c91-429b-ae21-2d225fab5031 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:09.165727 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 579/1160] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:26:09 2022] GET /identity/v3/projects/985bb364a7d24a9aa4fa40a6356071b1 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:09.172098 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b5c890-a54b-4ed3-8273-04db4bf1c60c None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:09.217234 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:09.269521 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b5c890-a54b-4ed3-8273-04db4bf1c60c None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:09.302571 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d9ccb-baff-495b-b1e5-cd269656e6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:09.303226 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d9ccb-baff-495b-b1e5-cd269656e6b7 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:09.303532 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d9ccb-baff-495b-b1e5-cd269656e6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:09.303864 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d9ccb-baff-495b-b1e5-cd269656e6b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:09.304698 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318d9ccb-baff-495b-b1e5-cd269656e6b7 None admin] RBAC: Adding query filter params (role.id=98eaadb3669f4b608d572c001742f559, scope.project.id=985bb364a7d24a9aa4fa40a6356071b1, user.id=c555a90289274f9989bd4e919a14c943) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:26:09.305013 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318d9ccb-baff-495b-b1e5-cd269656e6b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:09.306445 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318d9ccb-baff-495b-b1e5-cd269656e6b7 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:09.312215 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 582/1161] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 18 16:26:09 2022] GET /identity/v3/role_assignments?user.id=c555a90289274f9989bd4e919a14c943&scope.project.id=985bb364a7d24a9aa4fa40a6356071b1&role.id=98eaadb3669f4b608d572c001742f559 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:11.350463 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd52c1-333c-4a3b-94c3-2f0134865af1 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:11.350997 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd52c1-333c-4a3b-94c3-2f0134865af1 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:11.350997 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd52c1-333c-4a3b-94c3-2f0134865af1 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:11.352417 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 580/1162] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:26:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:26:11.357522 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186889bb-43a1-4b42-bd6c-7fb2a7e862c0 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:11.357905 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186889bb-43a1-4b42-bd6c-7fb2a7e862c0 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:11.358081 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186889bb-43a1-4b42-bd6c-7fb2a7e862c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:11.374886 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-186889bb-43a1-4b42-bd6c-7fb2a7e862c0 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:26:11.376097 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-186889bb-43a1-4b42-bd6c-7fb2a7e862c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:26:11.440900 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 583/1163] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:26:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:26:11.462740 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cd52c1-333c-4a3b-94c3-2f0134865af1 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:11.540739 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:11.563721 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cd52c1-333c-4a3b-94c3-2f0134865af1 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:11.597012 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa79c082-2412-4676-a7b9-7d2544ed55a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:11.597649 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79c082-2412-4676-a7b9-7d2544ed55a8 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:11.597929 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79c082-2412-4676-a7b9-7d2544ed55a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:11.598200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79c082-2412-4676-a7b9-7d2544ed55a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:11.663144 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa79c082-2412-4676-a7b9-7d2544ed55a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:11.664840 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa79c082-2412-4676-a7b9-7d2544ed55a8 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:11.739607 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:11.763662 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 581/1164] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:26:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 16:26:11.769644 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186889bb-43a1-4b42-bd6c-7fb2a7e862c0 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:11.811794 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:11.872234 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186889bb-43a1-4b42-bd6c-7fb2a7e862c0 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:11.907788 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e16466-3842-4e90-a991-2a048cf61bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:11.909278 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e16466-3842-4e90-a991-2a048cf61bcb None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:11.909564 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e16466-3842-4e90-a991-2a048cf61bcb None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:11.909840 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e16466-3842-4e90-a991-2a048cf61bcb None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:11.916979 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e16466-3842-4e90-a991-2a048cf61bcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:11.918362 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e16466-3842-4e90-a991-2a048cf61bcb None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:11.932898 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-58e16466-3842-4e90-a991-2a048cf61bcb None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 18 16:26:11.932898 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:26:11.932898 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 18 16:26:11.932898 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 18 16:26:11.932898 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:26:11.932898 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:26:11.932898 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:26:11.932898 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:26:11.932898 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 18 16:26:11.932898 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:26:11.932898 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:26:11.932898 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:26:11.932898 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:26:11.932898 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:26:11.932898 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:26:11.932898 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:26:11.932898 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:26:11.934091 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:26:11.934091 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:26:11.934091 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:26:11.934091 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:26:11.934091 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:26:11.934091 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:26:11.934091 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:26:11.934091 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:26:11.934091 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:26:11.934091 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:26:11.934091 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:26:11.934091 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:26:11.934091 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:26:11.934091 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:26:11.934091 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:26:11.934091 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:26:11.934951 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:26:11.934951 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:26:11.934951 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:26:11.934951 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:26:11.934951 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:26:11.934951 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:26:11.934951 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:26:11.934951 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:26:11.934951 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:26:11.934951 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:26:11.934951 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 18 16:26:11.934951 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:26:11.934951 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 18 16:26:11.934951 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:26:11.934951 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 584/1165] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Jul 18 16:26:11 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:26:11.941005 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa79c082-2412-4676-a7b9-7d2544ed55a8 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:12.042195 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa79c082-2412-4676-a7b9-7d2544ed55a8 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:12.081210 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b080d79b-d041-4b8f-828b-1d029e1d4f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:12.082026 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b080d79b-d041-4b8f-828b-1d029e1d4f97 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:12.082313 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b080d79b-d041-4b8f-828b-1d029e1d4f97 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:12.082591 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b080d79b-d041-4b8f-828b-1d029e1d4f97 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:12.083358 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b080d79b-d041-4b8f-828b-1d029e1d4f97 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:26:12.083732 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b080d79b-d041-4b8f-828b-1d029e1d4f97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:12.085287 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b080d79b-d041-4b8f-828b-1d029e1d4f97 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:12.094584 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 582/1166] 10.222.0.26 () {56 vars in 1115 bytes} [Mon Jul 18 16:26:11 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:14.141219 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2b6e1-5492-4660-ac53-0da9c885703e None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:14.141742 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2b6e1-5492-4660-ac53-0da9c885703e None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:14.141742 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2b6e1-5492-4660-ac53-0da9c885703e None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:14.143119 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 585/1167] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:26:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:26:14.148534 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4006483-132f-4e3d-b211-8da51e3adf70 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:14.149005 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4006483-132f-4e3d-b211-8da51e3adf70 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:14.149106 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4006483-132f-4e3d-b211-8da51e3adf70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:14.166844 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-e4006483-132f-4e3d-b211-8da51e3adf70 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:26:14.168192 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e4006483-132f-4e3d-b211-8da51e3adf70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:26:14.205801 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:14.239887 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 583/1168] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:26:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:26:14.263240 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c2b6e1-5492-4660-ac53-0da9c885703e None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:14.366280 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:14.370176 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c2b6e1-5492-4660-ac53-0da9c885703e None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:14.405222 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ffc588-4a56-442c-bb52-4beab985e1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:14.405822 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ffc588-4a56-442c-bb52-4beab985e1fc None admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:14.406109 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ffc588-4a56-442c-bb52-4beab985e1fc None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:14.406386 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ffc588-4a56-442c-bb52-4beab985e1fc None admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:14.407143 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ffc588-4a56-442c-bb52-4beab985e1fc None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:14.408616 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ffc588-4a56-442c-bb52-4beab985e1fc None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:14.493277 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 586/1169] 10.222.0.26 () {60 vars in 1138 bytes} [Mon Jul 18 16:26:14 2022] POST /identity/v3/domains => generated 267 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:26:16.574156 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b1d051-9666-48c3-935b-e8d279514123 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:16.574726 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b1d051-9666-48c3-935b-e8d279514123 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:16.574726 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b1d051-9666-48c3-935b-e8d279514123 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:16.576116 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 584/1170] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:26:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:26:16.581975 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d07676-aeaa-4a9e-966c-9d2528594430 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:16.582427 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d07676-aeaa-4a9e-966c-9d2528594430 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:16.582535 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d07676-aeaa-4a9e-966c-9d2528594430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:16.599085 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-28d07676-aeaa-4a9e-966c-9d2528594430 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:26:16.600338 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-28d07676-aeaa-4a9e-966c-9d2528594430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:26:16.666455 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 587/1171] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:26:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:26:16.690678 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b1d051-9666-48c3-935b-e8d279514123 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:16.789680 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b1d051-9666-48c3-935b-e8d279514123 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:16.824751 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af52723-0872-4cae-a3ba-0b84b9461b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:16.825533 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af52723-0872-4cae-a3ba-0b84b9461b30 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:16.825816 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af52723-0872-4cae-a3ba-0b84b9461b30 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:16.826092 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af52723-0872-4cae-a3ba-0b84b9461b30 None admin] PATH_INFO: `/v3/domains/fbd5863b9b734f9593101b2abaf7c4e1` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:16.836942 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af52723-0872-4cae-a3ba-0b84b9461b30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=fbd5863b9b734f9593101b2abaf7c4e1)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:16.838400 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af52723-0872-4cae-a3ba-0b84b9461b30 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:16.840146 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 585/1172] 10.222.0.26 () {56 vars in 1183 bytes} [Mon Jul 18 16:26:16 2022] GET /identity/v3/domains/fbd5863b9b734f9593101b2abaf7c4e1 => generated 267 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:16.846035 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d07676-aeaa-4a9e-966c-9d2528594430 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:16.934719 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:16.944889 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d07676-aeaa-4a9e-966c-9d2528594430 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:16.977555 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3914c7a0-6739-4ba4-91ae-ef4a5a00fcf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:16.978215 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3914c7a0-6739-4ba4-91ae-ef4a5a00fcf0 None admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:16.978497 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3914c7a0-6739-4ba4-91ae-ef4a5a00fcf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:16.978771 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3914c7a0-6739-4ba4-91ae-ef4a5a00fcf0 None admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:16.979484 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3914c7a0-6739-4ba4-91ae-ef4a5a00fcf0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:16.984392 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3914c7a0-6739-4ba4-91ae-ef4a5a00fcf0 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:17.076774 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 588/1173] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jul 18 16:26:16 2022] POST /identity/v3/projects => generated 361 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:26:19.147416 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c58bf-e1c9-4270-9a3a-d213e76b88b5 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:19.147994 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c58bf-e1c9-4270-9a3a-d213e76b88b5 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:19.147994 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c58bf-e1c9-4270-9a3a-d213e76b88b5 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:19.149303 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 586/1174] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:26:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:26:19.154582 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91489aae-de26-4958-9586-382dbc3a78f5 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:19.155036 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91489aae-de26-4958-9586-382dbc3a78f5 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:19.155141 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91489aae-de26-4958-9586-382dbc3a78f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:19.172630 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-91489aae-de26-4958-9586-382dbc3a78f5 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:26:19.174065 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-91489aae-de26-4958-9586-382dbc3a78f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:26:19.240195 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 589/1175] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:26:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:26:19.265521 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710c58bf-e1c9-4270-9a3a-d213e76b88b5 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:19.297685 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:19.368500 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710c58bf-e1c9-4270-9a3a-d213e76b88b5 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:19.403467 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bfce4c-c76d-4486-9d1f-688589c30cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:19.404264 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bfce4c-c76d-4486-9d1f-688589c30cc3 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:19.404547 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bfce4c-c76d-4486-9d1f-688589c30cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:19.404821 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bfce4c-c76d-4486-9d1f-688589c30cc3 None admin] PATH_INFO: `/v3/domains/fbd5863b9b734f9593101b2abaf7c4e1` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:19.405987 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bfce4c-c76d-4486-9d1f-688589c30cc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=fbd5863b9b734f9593101b2abaf7c4e1)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:19.407426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bfce4c-c76d-4486-9d1f-688589c30cc3 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:19.409359 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 587/1176] 10.222.0.26 () {56 vars in 1183 bytes} [Mon Jul 18 16:26:19 2022] GET /identity/v3/domains/fbd5863b9b734f9593101b2abaf7c4e1 => generated 267 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:19.415599 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91489aae-de26-4958-9586-382dbc3a78f5 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:19.489083 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:19.522408 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91489aae-de26-4958-9586-382dbc3a78f5 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:19.557066 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16e051e-4930-4c7f-8a34-112e36f4f8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:19.557778 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e051e-4930-4c7f-8a34-112e36f4f8fe None admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:19.558251 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e051e-4930-4c7f-8a34-112e36f4f8fe None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:19.558743 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e051e-4930-4c7f-8a34-112e36f4f8fe None admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:19.559939 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16e051e-4930-4c7f-8a34-112e36f4f8fe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:19.561404 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16e051e-4930-4c7f-8a34-112e36f4f8fe None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:19.650471 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 590/1177] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 18 16:26:19 2022] POST /identity/v3/users => generated 311 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:26:21.733180 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e60e442-1e99-4dab-b405-0eed2730b0b3 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:21.733731 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e60e442-1e99-4dab-b405-0eed2730b0b3 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:21.733731 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e60e442-1e99-4dab-b405-0eed2730b0b3 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:21.735132 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 588/1178] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:26:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:26:21.741982 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1b56d-e4dd-47f4-ae98-e19cf12a8de8 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:21.742543 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1b56d-e4dd-47f4-ae98-e19cf12a8de8 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:21.742543 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1b56d-e4dd-47f4-ae98-e19cf12a8de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:21.760157 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-e6d1b56d-e4dd-47f4-ae98-e19cf12a8de8 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:26:21.761353 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-e6d1b56d-e4dd-47f4-ae98-e19cf12a8de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:26:21.832196 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 591/1179] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:26:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:26:21.858703 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e60e442-1e99-4dab-b405-0eed2730b0b3 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:21.971136 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:21.976025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e60e442-1e99-4dab-b405-0eed2730b0b3 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:22.011867 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75a908c-d0f6-48c6-87da-0908615d6df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:22.012675 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a908c-d0f6-48c6-87da-0908615d6df4 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:22.012960 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a908c-d0f6-48c6-87da-0908615d6df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:22.013238 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a908c-d0f6-48c6-87da-0908615d6df4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:22.023873 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75a908c-d0f6-48c6-87da-0908615d6df4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:26:22.031549 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-e75a908c-d0f6-48c6-87da-0908615d6df4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:26:22.031549 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:26:22.031549 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:26:22.031549 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:26:22.031549 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:26:22.031549 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:26:22.031549 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:26:22.031549 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:26:22.031549 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:26:22.031549 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:26:22.031549 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:26:22.031549 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:26:22.031549 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:26:22.031549 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:26:22.031549 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:26:22.031549 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:26:22.033037 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:26:22.033037 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:26:22.033037 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:26:22.033037 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:26:22.033037 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:26:22.033037 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:26:22.033037 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:26:22.033037 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:26:22.033037 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:26:22.033037 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:26:22.033037 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:26:22.033037 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:26:22.033037 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:26:22.033037 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:26:22.033037 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:26:22.033037 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:26:22.034150 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:26:22.034150 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:26:22.034150 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:26:22.034150 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:26:22.034150 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:26:22.034150 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:26:22.034150 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:26:22.034150 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:26:22.034150 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:26:22.034150 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:26:22.034150 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:26:22.034150 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:26:22.034150 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 589/1180] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 18 16:26:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:26:22.039412 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d1b56d-e4dd-47f4-ae98-e19cf12a8de8 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:22.097890 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:22.139685 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d1b56d-e4dd-47f4-ae98-e19cf12a8de8 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:22.172895 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65b0124-0dfc-4341-9972-41960f709154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:22.174127 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b0124-0dfc-4341-9972-41960f709154 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:22.174407 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b0124-0dfc-4341-9972-41960f709154 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:22.174680 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b0124-0dfc-4341-9972-41960f709154 None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:22.175421 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65b0124-0dfc-4341-9972-41960f709154 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:26:22.175755 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65b0124-0dfc-4341-9972-41960f709154 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:22.177124 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65b0124-0dfc-4341-9972-41960f709154 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:22.185106 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 592/1181] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:26:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:22.191652 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75a908c-d0f6-48c6-87da-0908615d6df4 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:22.300532 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75a908c-d0f6-48c6-87da-0908615d6df4 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:22.337549 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e500773-b3cc-465a-ba86-7629b395d122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:22.338424 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e500773-b3cc-465a-ba86-7629b395d122 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:22.338706 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e500773-b3cc-465a-ba86-7629b395d122 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:22.338980 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e500773-b3cc-465a-ba86-7629b395d122 None admin] PATH_INFO: `/v3/users/89db7861a97a41bf8ddd001e0e0ff54f` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:22.352637 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e500773-b3cc-465a-ba86-7629b395d122 None admin] RBAC: Authorizing `identity:get_user(user_id=89db7861a97a41bf8ddd001e0e0ff54f)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:22.354086 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e500773-b3cc-465a-ba86-7629b395d122 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:22.355833 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 590/1182] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 18 16:26:22 2022] GET /identity/v3/users/89db7861a97a41bf8ddd001e0e0ff54f => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:22.361814 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65b0124-0dfc-4341-9972-41960f709154 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:22.466832 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65b0124-0dfc-4341-9972-41960f709154 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:22.500677 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:22.519216 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ef201d-ba73-4543-bfe5-33f1fcd6a976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:22.519914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef201d-ba73-4543-bfe5-33f1fcd6a976 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:22.520199 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef201d-ba73-4543-bfe5-33f1fcd6a976 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:22.520476 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef201d-ba73-4543-bfe5-33f1fcd6a976 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:22.590463 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ef201d-ba73-4543-bfe5-33f1fcd6a976 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:22.591887 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ef201d-ba73-4543-bfe5-33f1fcd6a976 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:22.692069 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 593/1183] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:26:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 18 16:26:22.697953 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e500773-b3cc-465a-ba86-7629b395d122 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:22.716230 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:22.798973 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e500773-b3cc-465a-ba86-7629b395d122 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:22.833396 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40972380-a728-4af2-bc7f-fc84f75360d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:22.834269 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40972380-a728-4af2-bc7f-fc84f75360d5 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:22.834550 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40972380-a728-4af2-bc7f-fc84f75360d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:22.834828 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40972380-a728-4af2-bc7f-fc84f75360d5 None admin] PATH_INFO: `/v3/projects/325b164a96964d3c9f778d7cb07df101` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:22.836191 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40972380-a728-4af2-bc7f-fc84f75360d5 None admin] RBAC: Authorizing `identity:get_project(project_id=325b164a96964d3c9f778d7cb07df101)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:22.837563 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40972380-a728-4af2-bc7f-fc84f75360d5 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:22.839370 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 591/1184] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:26:22 2022] GET /identity/v3/projects/325b164a96964d3c9f778d7cb07df101 => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:22.846519 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ef201d-ba73-4543-bfe5-33f1fcd6a976 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:22.868888 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:22.955120 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ef201d-ba73-4543-bfe5-33f1fcd6a976 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:22.992942 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabc6594-aab6-47e3-a9e3-be53c2430397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:22.993604 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabc6594-aab6-47e3-a9e3-be53c2430397 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:22.993884 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabc6594-aab6-47e3-a9e3-be53c2430397 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:22.994158 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabc6594-aab6-47e3-a9e3-be53c2430397 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:22.994991 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabc6594-aab6-47e3-a9e3-be53c2430397 None admin] RBAC: Adding query filter params (role.id=98eaadb3669f4b608d572c001742f559, scope.project.id=325b164a96964d3c9f778d7cb07df101, user.id=89db7861a97a41bf8ddd001e0e0ff54f) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:26:22.995296 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabc6594-aab6-47e3-a9e3-be53c2430397 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:22.996762 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabc6594-aab6-47e3-a9e3-be53c2430397 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:23.003739 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 594/1185] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 18 16:26:22 2022] GET /identity/v3/role_assignments?user.id=89db7861a97a41bf8ddd001e0e0ff54f&scope.project.id=325b164a96964d3c9f778d7cb07df101&role.id=98eaadb3669f4b608d572c001742f559 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:25.106849 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9675db-50fe-4b1c-8b12-7fcecb317060 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:25.107475 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9675db-50fe-4b1c-8b12-7fcecb317060 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:25.107475 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9675db-50fe-4b1c-8b12-7fcecb317060 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:25.108901 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 592/1186] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:26:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:26:25.115381 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b33ff1-927c-4475-a787-0b3daf6106fc None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:25.116031 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b33ff1-927c-4475-a787-0b3daf6106fc None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:25.116191 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b33ff1-927c-4475-a787-0b3daf6106fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:25.134959 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-21b33ff1-927c-4475-a787-0b3daf6106fc None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:26:25.136290 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-21b33ff1-927c-4475-a787-0b3daf6106fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:26:25.213648 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 595/1187] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:26:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:26:25.238633 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9675db-50fe-4b1c-8b12-7fcecb317060 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:25.333940 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:25.351656 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9675db-50fe-4b1c-8b12-7fcecb317060 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:25.393269 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93289b3b-6016-419b-b70b-cd15d05d9d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:25.393845 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93289b3b-6016-419b-b70b-cd15d05d9d34 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:25.394132 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93289b3b-6016-419b-b70b-cd15d05d9d34 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:25.394413 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93289b3b-6016-419b-b70b-cd15d05d9d34 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:25.405799 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93289b3b-6016-419b-b70b-cd15d05d9d34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:26:25.414167 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-93289b3b-6016-419b-b70b-cd15d05d9d34 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:26:25.414167 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:26:25.414167 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:26:25.414167 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:26:25.414167 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:26:25.414167 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:26:25.414167 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:26:25.414167 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:26:25.414167 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:26:25.414167 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:26:25.414167 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:26:25.414167 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:26:25.414167 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:26:25.414167 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:26:25.414167 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:26:25.414167 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:26:25.415987 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:26:25.415987 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:26:25.415987 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:26:25.415987 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:26:25.415987 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:26:25.415987 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:26:25.415987 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:26:25.415987 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:26:25.415987 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:26:25.415987 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:26:25.415987 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:26:25.415987 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:26:25.415987 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:26:25.415987 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:26:25.415987 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:26:25.415987 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:26:25.417747 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:26:25.417747 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:26:25.417747 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:26:25.417747 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:26:25.417747 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:26:25.417747 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:26:25.417747 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:26:25.417747 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:26:25.417747 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:26:25.417747 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:26:25.417747 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:26:25.417747 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:26:25.417747 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 593/1188] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 18 16:26:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:26:25.424004 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b33ff1-927c-4475-a787-0b3daf6106fc None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:25.441695 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:25.542230 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b33ff1-927c-4475-a787-0b3daf6106fc None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:25.581420 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd0ba4e-102f-4f9d-8666-eb824ce936e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:25.582100 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd0ba4e-102f-4f9d-8666-eb824ce936e3 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:25.582388 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd0ba4e-102f-4f9d-8666-eb824ce936e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:25.582668 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd0ba4e-102f-4f9d-8666-eb824ce936e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:25.583428 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd0ba4e-102f-4f9d-8666-eb824ce936e3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:26:25.583765 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd0ba4e-102f-4f9d-8666-eb824ce936e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:25.585156 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd0ba4e-102f-4f9d-8666-eb824ce936e3 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:25.594299 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 596/1189] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:26:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:26:25.600974 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93289b3b-6016-419b-b70b-cd15d05d9d34 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:25.717565 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93289b3b-6016-419b-b70b-cd15d05d9d34 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:25.755396 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe1085d-9fd9-48c4-9a97-71876321e1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:25.756306 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe1085d-9fd9-48c4-9a97-71876321e1c9 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:25.756591 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe1085d-9fd9-48c4-9a97-71876321e1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:25.756874 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe1085d-9fd9-48c4-9a97-71876321e1c9 None admin] PATH_INFO: `/v3/users/89db7861a97a41bf8ddd001e0e0ff54f` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:25.758095 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe1085d-9fd9-48c4-9a97-71876321e1c9 None admin] RBAC: Authorizing `identity:get_user(user_id=89db7861a97a41bf8ddd001e0e0ff54f)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:25.759571 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe1085d-9fd9-48c4-9a97-71876321e1c9 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:25.761247 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 594/1190] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 18 16:26:25 2022] GET /identity/v3/users/89db7861a97a41bf8ddd001e0e0ff54f => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:25.770335 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd0ba4e-102f-4f9d-8666-eb824ce936e3 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:25.869410 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:25.893680 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd0ba4e-102f-4f9d-8666-eb824ce936e3 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:25.930937 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e0380f-40aa-469c-ab2c-abc99ce71f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:25.931843 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e0380f-40aa-469c-ab2c-abc99ce71f85 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:25.932126 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e0380f-40aa-469c-ab2c-abc99ce71f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:25.933235 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e0380f-40aa-469c-ab2c-abc99ce71f85 None admin] PATH_INFO: `/v3/projects/325b164a96964d3c9f778d7cb07df101` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:25.934480 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e0380f-40aa-469c-ab2c-abc99ce71f85 None admin] RBAC: Authorizing `identity:get_project(project_id=325b164a96964d3c9f778d7cb07df101)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:25.935987 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e0380f-40aa-469c-ab2c-abc99ce71f85 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:25.937755 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 597/1191] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:26:25 2022] GET /identity/v3/projects/325b164a96964d3c9f778d7cb07df101 => generated 361 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:25.944454 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe1085d-9fd9-48c4-9a97-71876321e1c9 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:25.961761 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:26.053894 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe1085d-9fd9-48c4-9a97-71876321e1c9 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:26.090469 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707801e8-9f0b-4938-9f37-3233d114d864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:26.091367 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707801e8-9f0b-4938-9f37-3233d114d864 None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:26.091682 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707801e8-9f0b-4938-9f37-3233d114d864 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:26.091970 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707801e8-9f0b-4938-9f37-3233d114d864 None admin] PATH_INFO: `/v3/projects/325b164a96964d3c9f778d7cb07df101/users/89db7861a97a41bf8ddd001e0e0ff54f/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:26.094040 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707801e8-9f0b-4938-9f37-3233d114d864 None admin] RBAC: Authorizing `identity:create_grant(project_id=325b164a96964d3c9f778d7cb07df101, user_id=89db7861a97a41bf8ddd001e0e0ff54f, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:26.096626 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707801e8-9f0b-4938-9f37-3233d114d864 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:26.207793 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 595/1192] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 18 16:26:25 2022] PUT /identity/v3/projects/325b164a96964d3c9f778d7cb07df101/users/89db7861a97a41bf8ddd001e0e0ff54f/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:26:28.291015 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7466e7-f7af-411f-ba5e-804fd3c7c5f7 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:28.291601 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7466e7-f7af-411f-ba5e-804fd3c7c5f7 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:28.291601 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7466e7-f7af-411f-ba5e-804fd3c7c5f7 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:28.292980 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 598/1193] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:26:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:26:28.298943 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7256c32-ce58-418f-8514-fc74f3309c5b None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:28.299462 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7256c32-ce58-418f-8514-fc74f3309c5b None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:28.299585 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7256c32-ce58-418f-8514-fc74f3309c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:28.317306 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-a7256c32-ce58-418f-8514-fc74f3309c5b None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:26:28.318533 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a7256c32-ce58-418f-8514-fc74f3309c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:26:28.391188 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 596/1194] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:26:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:26:28.416708 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7466e7-f7af-411f-ba5e-804fd3c7c5f7 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:28.580480 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7466e7-f7af-411f-ba5e-804fd3c7c5f7 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:28.625746 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78e484f-71b4-4d61-bbd7-fa8c8fff21c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:28.626874 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78e484f-71b4-4d61-bbd7-fa8c8fff21c3 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:28.627170 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78e484f-71b4-4d61-bbd7-fa8c8fff21c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:28.627455 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78e484f-71b4-4d61-bbd7-fa8c8fff21c3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:28.639266 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78e484f-71b4-4d61-bbd7-fa8c8fff21c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:26:28.647802 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-a78e484f-71b4-4d61-bbd7-fa8c8fff21c3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:26:28.647802 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:26:28.647802 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:26:28.647802 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:26:28.647802 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:26:28.647802 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:26:28.647802 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:26:28.647802 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:26:28.647802 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:26:28.647802 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:26:28.647802 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:26:28.647802 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:26:28.647802 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:26:28.647802 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:26:28.647802 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:26:28.647802 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:26:28.648914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:26:28.648914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:26:28.648914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:26:28.648914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:26:28.648914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:26:28.648914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:26:28.648914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:26:28.648914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:26:28.648914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:26:28.648914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:26:28.648914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:26:28.648914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:26:28.648914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:26:28.648914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:26:28.648914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:26:28.648914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:26:28.649836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:26:28.649836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:26:28.649836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:26:28.649836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:26:28.649836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:26:28.649836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:26:28.649836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:26:28.649836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:26:28.649836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:26:28.649836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:26:28.649836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:26:28.649836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:26:28.649836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 599/1195] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 18 16:26:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:26:28.655552 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7256c32-ce58-418f-8514-fc74f3309c5b None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:28.658985 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:28.768264 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7256c32-ce58-418f-8514-fc74f3309c5b None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:28.803126 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28cf675-1b2c-4447-beae-35991ac6569e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:28.803823 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28cf675-1b2c-4447-beae-35991ac6569e None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:28.804112 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28cf675-1b2c-4447-beae-35991ac6569e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:28.804391 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28cf675-1b2c-4447-beae-35991ac6569e None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:28.805141 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28cf675-1b2c-4447-beae-35991ac6569e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:26:28.805449 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28cf675-1b2c-4447-beae-35991ac6569e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:28.806817 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28cf675-1b2c-4447-beae-35991ac6569e None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:28.815812 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 597/1196] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:26:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:28.821804 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78e484f-71b4-4d61-bbd7-fa8c8fff21c3 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:28.825637 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:28.925999 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78e484f-71b4-4d61-bbd7-fa8c8fff21c3 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:28.959511 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6797f0-88e2-472f-95e9-10e688059d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:28.960404 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6797f0-88e2-472f-95e9-10e688059d72 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:28.960687 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6797f0-88e2-472f-95e9-10e688059d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:28.960961 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6797f0-88e2-472f-95e9-10e688059d72 None admin] PATH_INFO: `/v3/users/89db7861a97a41bf8ddd001e0e0ff54f` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:28.962156 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6797f0-88e2-472f-95e9-10e688059d72 None admin] RBAC: Authorizing `identity:get_user(user_id=89db7861a97a41bf8ddd001e0e0ff54f)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:28.963605 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6797f0-88e2-472f-95e9-10e688059d72 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:28.965247 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 600/1197] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 18 16:26:28 2022] GET /identity/v3/users/89db7861a97a41bf8ddd001e0e0ff54f => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:28.971275 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28cf675-1b2c-4447-beae-35991ac6569e None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:29.035100 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:29.073055 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28cf675-1b2c-4447-beae-35991ac6569e None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:29.106919 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a01be-640d-4594-8fe8-73f8050c0e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:29.107483 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a01be-640d-4594-8fe8-73f8050c0e03 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:29.107799 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a01be-640d-4594-8fe8-73f8050c0e03 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:29.108081 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a01be-640d-4594-8fe8-73f8050c0e03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:29.178487 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479a01be-640d-4594-8fe8-73f8050c0e03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:29.179977 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479a01be-640d-4594-8fe8-73f8050c0e03 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:29.240045 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:29.276957 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 598/1198] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:26:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 16:26:29.283689 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6797f0-88e2-472f-95e9-10e688059d72 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:29.354970 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:29.385823 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6797f0-88e2-472f-95e9-10e688059d72 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:29.423341 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b202c-d22e-457e-a945-a46feeeb9650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:29.424292 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b202c-d22e-457e-a945-a46feeeb9650 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:29.424577 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b202c-d22e-457e-a945-a46feeeb9650 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:29.424854 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b202c-d22e-457e-a945-a46feeeb9650 None admin] PATH_INFO: `/v3/projects/325b164a96964d3c9f778d7cb07df101` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:29.426128 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3b202c-d22e-457e-a945-a46feeeb9650 None admin] RBAC: Authorizing `identity:get_project(project_id=325b164a96964d3c9f778d7cb07df101)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:29.427625 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3b202c-d22e-457e-a945-a46feeeb9650 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:29.429395 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 601/1199] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 18 16:26:29 2022] GET /identity/v3/projects/325b164a96964d3c9f778d7cb07df101 => generated 361 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:29.435770 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a01be-640d-4594-8fe8-73f8050c0e03 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:26:29.536407 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a01be-640d-4594-8fe8-73f8050c0e03 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:26:29.571126 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497dc5a7-de68-4f4c-8b5b-99107e8c2af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:26:29.571803 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497dc5a7-de68-4f4c-8b5b-99107e8c2af4 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:26:29.572089 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497dc5a7-de68-4f4c-8b5b-99107e8c2af4 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:26:29.572366 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497dc5a7-de68-4f4c-8b5b-99107e8c2af4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:26:29.573196 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497dc5a7-de68-4f4c-8b5b-99107e8c2af4 None admin] RBAC: Adding query filter params (role.id=98eaadb3669f4b608d572c001742f559, scope.project.id=325b164a96964d3c9f778d7cb07df101, user.id=89db7861a97a41bf8ddd001e0e0ff54f) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:26:29.573509 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497dc5a7-de68-4f4c-8b5b-99107e8c2af4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:26:29.574946 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497dc5a7-de68-4f4c-8b5b-99107e8c2af4 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:26:29.581153 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 599/1200] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 18 16:26:29 2022] GET /identity/v3/role_assignments?user.id=89db7861a97a41bf8ddd001e0e0ff54f&scope.project.id=325b164a96964d3c9f778d7cb07df101&role.id=98eaadb3669f4b608d572c001742f559 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:26:39.418403 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=30 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:26:39.578186 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=86 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:27:50.865972 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f5464d-ef1f-4340-913b-b090889144bc None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:27:50.866511 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f5464d-ef1f-4340-913b-b090889144bc None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:27:50.866511 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f5464d-ef1f-4340-913b-b090889144bc None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:27:50.867934 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 602/1201] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:27:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:27:50.873633 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03c16c0-3ff1-45e8-89e6-1f70e77081db None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:27:50.874437 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03c16c0-3ff1-45e8-89e6-1f70e77081db None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:27:50.874716 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03c16c0-3ff1-45e8-89e6-1f70e77081db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:27:50.892408 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-b03c16c0-3ff1-45e8-89e6-1f70e77081db None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:27:50.893622 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-b03c16c0-3ff1-45e8-89e6-1f70e77081db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:27:50.960535 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 600/1202] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:27:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:27:50.986027 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f5464d-ef1f-4340-913b-b090889144bc None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:27:51.088793 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f5464d-ef1f-4340-913b-b090889144bc None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:27:51.122615 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e3566-930a-4a1e-be83-849b02901d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:27:51.124142 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e3566-930a-4a1e-be83-849b02901d40 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:27:51.124443 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e3566-930a-4a1e-be83-849b02901d40 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:27:51.124751 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e3566-930a-4a1e-be83-849b02901d40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:27:51.153569 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9e3566-930a-4a1e-be83-849b02901d40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:27:51.155133 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9e3566-930a-4a1e-be83-849b02901d40 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:27:51.174841 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-5e9e3566-930a-4a1e-be83-849b02901d40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:27:51.174841 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:27:51.174841 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:27:51.174841 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:27:51.174841 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:27:51.174841 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:27:51.174841 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:27:51.174841 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:27:51.174841 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:27:51.174841 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:27:51.174841 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:27:51.174841 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:27:51.174841 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:27:51.174841 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:27:51.174841 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:27:51.176071 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:27:51.176071 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:27:51.176071 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:27:51.176071 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:27:51.176071 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:27:51.176071 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:27:51.176071 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:27:51.176071 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:27:51.176071 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:27:51.176071 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:27:51.176071 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:27:51.176071 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:27:51.176071 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:27:51.176071 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:27:51.176071 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:27:51.176071 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:27:51.176071 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:27:51.177249 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:27:51.177249 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:27:51.177249 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:27:51.177249 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:27:51.177249 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 603/1203] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:27:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:27:51.182818 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03c16c0-3ff1-45e8-89e6-1f70e77081db None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:27:51.299512 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03c16c0-3ff1-45e8-89e6-1f70e77081db None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:27:51.329750 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:27:51.335616 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f2368f-1832-4aa3-ab62-4c8df21038a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:27:51.336295 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2368f-1832-4aa3-ab62-4c8df21038a2 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:27:51.336576 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2368f-1832-4aa3-ab62-4c8df21038a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:27:51.336850 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2368f-1832-4aa3-ab62-4c8df21038a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:27:51.337579 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f2368f-1832-4aa3-ab62-4c8df21038a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:27:51.337881 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f2368f-1832-4aa3-ab62-4c8df21038a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:27:51.339325 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f2368f-1832-4aa3-ab62-4c8df21038a2 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:27:51.351898 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 601/1204] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:27:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:27:51.358369 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e3566-930a-4a1e-be83-849b02901d40 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:27:51.416725 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:27:51.469388 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e3566-930a-4a1e-be83-849b02901d40 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:27:51.506258 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b9ceee-c2e3-4a42-be8b-b9adb5f0b5a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:27:51.506258 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9ceee-c2e3-4a42-be8b-b9adb5f0b5a5 None admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:27:51.506650 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9ceee-c2e3-4a42-be8b-b9adb5f0b5a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:27:51.507135 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9ceee-c2e3-4a42-be8b-b9adb5f0b5a5 None admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:27:51.507544 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b9ceee-c2e3-4a42-be8b-b9adb5f0b5a5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:27:51.508983 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b9ceee-c2e3-4a42-be8b-b9adb5f0b5a5 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:27:51.611811 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 604/1205] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 18 16:27:51 2022] POST /identity/v3/users => generated 251 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:27:53.766443 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d9f1b-f06a-4bcc-be72-fb47d967a4e2 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:27:53.767044 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d9f1b-f06a-4bcc-be72-fb47d967a4e2 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:27:53.767044 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d9f1b-f06a-4bcc-be72-fb47d967a4e2 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:27:53.768535 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 602/1206] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:27:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:27:53.773861 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80090b33-a560-44a3-a6ae-d4625244b148 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:27:53.774329 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80090b33-a560-44a3-a6ae-d4625244b148 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:27:53.774436 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80090b33-a560-44a3-a6ae-d4625244b148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:27:53.791676 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-80090b33-a560-44a3-a6ae-d4625244b148 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:27:53.792848 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-80090b33-a560-44a3-a6ae-d4625244b148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:27:53.862477 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 605/1207] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:27:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:27:53.887832 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d9f1b-f06a-4bcc-be72-fb47d967a4e2 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:27:53.993502 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d9f1b-f06a-4bcc-be72-fb47d967a4e2 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:27:54.027979 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4db244-9abc-4500-bcbe-a38267026f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:27:54.028670 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4db244-9abc-4500-bcbe-a38267026f48 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:27:54.028986 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4db244-9abc-4500-bcbe-a38267026f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:27:54.029240 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4db244-9abc-4500-bcbe-a38267026f48 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:27:54.040465 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4db244-9abc-4500-bcbe-a38267026f48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:27:54.048416 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-0c4db244-9abc-4500-bcbe-a38267026f48 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:27:54.048416 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:27:54.048416 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:27:54.048416 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:27:54.048416 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:27:54.048416 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:27:54.048416 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:27:54.048416 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:27:54.048416 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:27:54.048416 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:27:54.048416 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:27:54.048416 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:27:54.048416 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:27:54.048416 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:27:54.048416 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:27:54.048416 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:27:54.049410 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:27:54.049410 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:27:54.049410 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:27:54.049410 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:27:54.049410 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:27:54.049410 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:27:54.049410 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:27:54.049410 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:27:54.049410 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:27:54.049410 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:27:54.049410 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:27:54.049410 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:27:54.049410 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:27:54.049410 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:27:54.049410 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:27:54.049410 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:27:54.050293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:27:54.050293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:27:54.050293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:27:54.050293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:27:54.050293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:27:54.050293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:27:54.050293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:27:54.050293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:27:54.050293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:27:54.050293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:27:54.050293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:27:54.050293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:27:54.050293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 603/1208] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 18 16:27:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:27:54.055910 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80090b33-a560-44a3-a6ae-d4625244b148 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:27:54.097398 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:27:54.156006 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80090b33-a560-44a3-a6ae-d4625244b148 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:27:54.189734 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49802e0-e422-48bb-b02c-adce4dd66a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:27:54.190411 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49802e0-e422-48bb-b02c-adce4dd66a0f None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:27:54.190695 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49802e0-e422-48bb-b02c-adce4dd66a0f None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:27:54.190971 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49802e0-e422-48bb-b02c-adce4dd66a0f None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:27:54.191749 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49802e0-e422-48bb-b02c-adce4dd66a0f None admin] RBAC: Adding query filter params (name=service) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:27:54.192064 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49802e0-e422-48bb-b02c-adce4dd66a0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:27:54.193429 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49802e0-e422-48bb-b02c-adce4dd66a0f None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:27:54.201616 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 606/1209] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 18 16:27:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:27:54.208058 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4db244-9abc-4500-bcbe-a38267026f48 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:27:54.265565 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:27:54.319844 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4db244-9abc-4500-bcbe-a38267026f48 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:27:54.356142 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49dbf16-04f5-4bd9-9de8-532a2c1d3235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:27:54.356962 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49dbf16-04f5-4bd9-9de8-532a2c1d3235 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:27:54.357218 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49dbf16-04f5-4bd9-9de8-532a2c1d3235 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:27:54.358591 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49dbf16-04f5-4bd9-9de8-532a2c1d3235 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:27:54.369569 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49dbf16-04f5-4bd9-9de8-532a2c1d3235 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:27:54.371020 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49dbf16-04f5-4bd9-9de8-532a2c1d3235 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:27:54.389477 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-b49dbf16-04f5-4bd9-9de8-532a2c1d3235 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:27:54.389477 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:27:54.389477 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:27:54.389477 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:27:54.389477 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:27:54.389477 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:27:54.389477 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:27:54.389477 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:27:54.389477 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:27:54.389477 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:27:54.389477 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:27:54.389477 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:27:54.389477 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:27:54.389477 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:27:54.389477 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:27:54.391831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:27:54.391831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:27:54.391831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:27:54.391831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:27:54.391831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:27:54.391831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:27:54.391831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:27:54.391831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:27:54.391831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:27:54.391831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:27:54.391831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:27:54.391831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:27:54.391831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:27:54.391831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:27:54.391831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:27:54.391831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:27:54.391831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:27:54.392924 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:27:54.392924 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:27:54.392924 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:27:54.392924 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:27:54.392924 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 604/1210] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:27:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:27:54.396717 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49802e0-e422-48bb-b02c-adce4dd66a0f None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:27:54.494890 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:27:54.498600 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49802e0-e422-48bb-b02c-adce4dd66a0f None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:27:54.531339 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438fe144-bc54-467f-8a78-b990da11bf1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:27:54.532112 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438fe144-bc54-467f-8a78-b990da11bf1d None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:27:54.532405 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438fe144-bc54-467f-8a78-b990da11bf1d None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:27:54.532782 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438fe144-bc54-467f-8a78-b990da11bf1d None admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:27:54.533423 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438fe144-bc54-467f-8a78-b990da11bf1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:27:54.533726 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438fe144-bc54-467f-8a78-b990da11bf1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:27:54.535112 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438fe144-bc54-467f-8a78-b990da11bf1d None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:27:54.547256 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 607/1211] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:27:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:27:54.553125 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49dbf16-04f5-4bd9-9de8-532a2c1d3235 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:27:54.657372 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49dbf16-04f5-4bd9-9de8-532a2c1d3235 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:27:54.659331 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:27:54.702623 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4de30b-b29d-4516-8639-719a38ce35ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:27:54.703428 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4de30b-b29d-4516-8639-719a38ce35ed None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:27:54.703742 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4de30b-b29d-4516-8639-719a38ce35ed None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:27:54.704012 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4de30b-b29d-4516-8639-719a38ce35ed None admin] PATH_INFO: `/v3/users/placement` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:27:54.711140 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4de30b-b29d-4516-8639-719a38ce35ed None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:27:54.715341 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4de30b-b29d-4516-8639-719a38ce35ed None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:27:54.727364 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-8a4de30b-b29d-4516-8639-719a38ce35ed None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 16:27:54.727364 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:27:54.727364 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:27:54.727364 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:27:54.727364 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:27:54.727364 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:27:54.727364 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:27:54.727364 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:27:54.727364 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:27:54.727364 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:27:54.727364 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:27:54.727364 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:27:54.727364 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:27:54.727364 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:27:54.727364 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:27:54.727364 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:27:54.729223 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:27:54.729223 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:27:54.729223 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:27:54.729223 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:27:54.729223 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:27:54.729223 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:27:54.729223 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:27:54.729223 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:27:54.729223 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:27:54.729223 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:27:54.729223 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:27:54.729223 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:27:54.729223 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:27:54.729223 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:27:54.729223 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:27:54.729223 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:27:54.729223 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:27:54.729223 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:27:54.731059 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:27:54.731059 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:27:54.731059 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:27:54.731059 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:27:54.731059 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:27:54.731059 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:27:54.731059 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:27:54.731059 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:27:54.731059 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 16:27:54.731059 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:27:54.731059 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 605/1212] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:27:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:27:54.734956 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438fe144-bc54-467f-8a78-b990da11bf1d None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:27:54.840763 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438fe144-bc54-467f-8a78-b990da11bf1d None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:27:54.875450 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01520dc-1886-44de-9962-9fee8ee29d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:27:54.876160 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01520dc-1886-44de-9962-9fee8ee29d0e None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:27:54.876445 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01520dc-1886-44de-9962-9fee8ee29d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:27:54.876756 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01520dc-1886-44de-9962-9fee8ee29d0e None admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:27:54.877645 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01520dc-1886-44de-9962-9fee8ee29d0e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:27:54.877954 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01520dc-1886-44de-9962-9fee8ee29d0e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:27:54.879383 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01520dc-1886-44de-9962-9fee8ee29d0e None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:27:54.897274 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 608/1213] 10.222.0.26 () {56 vars in 1143 bytes} [Mon Jul 18 16:27:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:27:54.903218 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4de30b-b29d-4516-8639-719a38ce35ed None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:27:55.005185 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4de30b-b29d-4516-8639-719a38ce35ed None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:27:55.041148 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e37d28-bdc8-47ac-b7a8-2ab189fd6db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:27:55.041939 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e37d28-bdc8-47ac-b7a8-2ab189fd6db4 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:27:55.042229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e37d28-bdc8-47ac-b7a8-2ab189fd6db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:27:55.042511 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e37d28-bdc8-47ac-b7a8-2ab189fd6db4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:27:55.073708 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:27:55.115196 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e37d28-bdc8-47ac-b7a8-2ab189fd6db4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:27:55.116726 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e37d28-bdc8-47ac-b7a8-2ab189fd6db4 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:27:55.220722 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 606/1214] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:27:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 16:27:55.226680 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01520dc-1886-44de-9962-9fee8ee29d0e None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:27:55.246342 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:27:55.328208 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01520dc-1886-44de-9962-9fee8ee29d0e None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:27:55.361628 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe986d2-df6a-4a2f-b1fb-40bae12a6fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:27:55.363209 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe986d2-df6a-4a2f-b1fb-40bae12a6fce None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:27:55.363209 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe986d2-df6a-4a2f-b1fb-40bae12a6fce None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:27:55.363209 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe986d2-df6a-4a2f-b1fb-40bae12a6fce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:27:55.372771 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe986d2-df6a-4a2f-b1fb-40bae12a6fce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:27:55.374448 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe986d2-df6a-4a2f-b1fb-40bae12a6fce None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:27:55.392082 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-dfe986d2-df6a-4a2f-b1fb-40bae12a6fce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:27:55.392082 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:27:55.392082 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:27:55.392082 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:27:55.392082 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:27:55.392082 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:27:55.392082 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:27:55.392082 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:27:55.392082 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:27:55.392082 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:27:55.392082 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:27:55.392082 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:27:55.392082 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:27:55.392082 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:27:55.392082 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:27:55.393177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:27:55.393177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:27:55.393177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:27:55.393177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:27:55.393177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:27:55.393177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:27:55.393177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:27:55.393177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:27:55.393177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:27:55.393177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:27:55.393177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:27:55.393177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:27:55.393177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:27:55.393177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:27:55.393177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:27:55.393177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:27:55.393177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:27:55.394660 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:27:55.394660 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:27:55.394660 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:27:55.394660 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:27:55.394660 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 609/1215] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:27:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:27:55.399311 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e37d28-bdc8-47ac-b7a8-2ab189fd6db4 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:27:55.461188 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:27:55.501092 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e37d28-bdc8-47ac-b7a8-2ab189fd6db4 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:27:55.534159 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9486500b-b229-4d20-9bfb-5026bdcbf60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:27:55.534825 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486500b-b229-4d20-9bfb-5026bdcbf60c None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:27:55.535109 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486500b-b229-4d20-9bfb-5026bdcbf60c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:27:55.535383 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486500b-b229-4d20-9bfb-5026bdcbf60c None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:27:55.536141 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9486500b-b229-4d20-9bfb-5026bdcbf60c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:27:55.536448 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9486500b-b229-4d20-9bfb-5026bdcbf60c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:27:55.537818 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9486500b-b229-4d20-9bfb-5026bdcbf60c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:27:55.550755 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 607/1216] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:27:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:27:55.556572 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe986d2-df6a-4a2f-b1fb-40bae12a6fce None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:27:55.625256 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:27:55.657663 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe986d2-df6a-4a2f-b1fb-40bae12a6fce None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:27:55.693230 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3502f7-7bc1-456f-b26f-fae5dfed0aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:27:55.694029 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3502f7-7bc1-456f-b26f-fae5dfed0aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:27:55.694317 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3502f7-7bc1-456f-b26f-fae5dfed0aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:27:55.694595 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3502f7-7bc1-456f-b26f-fae5dfed0aa2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:27:55.701492 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3502f7-7bc1-456f-b26f-fae5dfed0aa2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:27:55.702940 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3502f7-7bc1-456f-b26f-fae5dfed0aa2 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:27:55.717525 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-8b3502f7-7bc1-456f-b26f-fae5dfed0aa2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:27:55.717525 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:27:55.717525 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:27:55.717525 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:27:55.717525 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:27:55.717525 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:27:55.717525 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:27:55.717525 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:27:55.717525 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:27:55.717525 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:27:55.717525 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:27:55.717525 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:27:55.717525 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:27:55.717525 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:27:55.717525 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:27:55.719021 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:27:55.719021 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:27:55.719021 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:27:55.719021 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:27:55.719021 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:27:55.719021 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:27:55.719021 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:27:55.719021 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:27:55.719021 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:27:55.719021 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:27:55.719021 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:27:55.719021 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:27:55.719021 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:27:55.719021 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:27:55.719021 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:27:55.719021 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:27:55.719021 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:27:55.720231 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:27:55.720231 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:27:55.720231 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:27:55.720231 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:27:55.720231 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:27:55.720231 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:27:55.720231 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:27:55.720231 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:27:55.720231 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 610/1217] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:27:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:27:55.724729 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9486500b-b229-4d20-9bfb-5026bdcbf60c None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:27:55.824867 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9486500b-b229-4d20-9bfb-5026bdcbf60c None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:27:55.850183 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:27:55.862193 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fa4bc0-b093-422e-8cc6-b4ac4e6a4e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:27:55.862850 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa4bc0-b093-422e-8cc6-b4ac4e6a4e56 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:27:55.863129 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa4bc0-b093-422e-8cc6-b4ac4e6a4e56 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:27:55.863404 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa4bc0-b093-422e-8cc6-b4ac4e6a4e56 None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:27:55.864210 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fa4bc0-b093-422e-8cc6-b4ac4e6a4e56 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:27:55.864513 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fa4bc0-b093-422e-8cc6-b4ac4e6a4e56 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:27:55.865929 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fa4bc0-b093-422e-8cc6-b4ac4e6a4e56 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:27:55.877856 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 608/1218] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 18 16:27:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:27:55.883888 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3502f7-7bc1-456f-b26f-fae5dfed0aa2 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:27:55.985318 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3502f7-7bc1-456f-b26f-fae5dfed0aa2 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:27:56.015028 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:27:56.020733 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27046dd-2349-45a5-8c6a-6d3171894608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:27:56.021386 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27046dd-2349-45a5-8c6a-6d3171894608 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:27:56.021671 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27046dd-2349-45a5-8c6a-6d3171894608 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:27:56.021950 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27046dd-2349-45a5-8c6a-6d3171894608 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:27:56.022782 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27046dd-2349-45a5-8c6a-6d3171894608 None admin] RBAC: Adding query filter params (role.id=4fe970e98f394bbfb096c4879bebdf97, scope.project.id=eb9747e2299c4e0991fd7edb4462c7e5, user.id=70f704a231a5474e8067a6c2dd88b355) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:27:56.023093 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27046dd-2349-45a5-8c6a-6d3171894608 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:27:56.024570 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27046dd-2349-45a5-8c6a-6d3171894608 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:27:56.030490 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 611/1219] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 18 16:27:55 2022] GET /identity/v3/role_assignments?user.id=70f704a231a5474e8067a6c2dd88b355&scope.project.id=eb9747e2299c4e0991fd7edb4462c7e5&role.id=4fe970e98f394bbfb096c4879bebdf97 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:27:58.052739 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0bca27-b77a-441d-bfba-10425cda3366 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:27:58.053286 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0bca27-b77a-441d-bfba-10425cda3366 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:27:58.053286 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0bca27-b77a-441d-bfba-10425cda3366 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:27:58.054654 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 609/1220] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:27:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:27:58.059902 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f6cd0e-2b58-4c03-bce9-734056c87af2 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:27:58.060345 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f6cd0e-2b58-4c03-bce9-734056c87af2 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:27:58.060456 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f6cd0e-2b58-4c03-bce9-734056c87af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:27:58.077953 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-c5f6cd0e-2b58-4c03-bce9-734056c87af2 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:27:58.079158 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-c5f6cd0e-2b58-4c03-bce9-734056c87af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:27:58.146542 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 612/1221] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:27:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:27:58.169509 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0bca27-b77a-441d-bfba-10425cda3366 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:27:58.275708 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0bca27-b77a-441d-bfba-10425cda3366 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:27:58.309856 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa87fda4-970f-487e-b9c5-8c7b731cb843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:27:58.310753 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa87fda4-970f-487e-b9c5-8c7b731cb843 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:27:58.311009 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa87fda4-970f-487e-b9c5-8c7b731cb843 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:27:58.311284 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa87fda4-970f-487e-b9c5-8c7b731cb843 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:27:58.321137 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa87fda4-970f-487e-b9c5-8c7b731cb843 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:27:58.329380 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-aa87fda4-970f-487e-b9c5-8c7b731cb843 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:27:58.329380 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:27:58.329380 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:27:58.329380 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:27:58.329380 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:27:58.329380 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:27:58.329380 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:27:58.329380 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:27:58.329380 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:27:58.329380 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:27:58.329380 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:27:58.329380 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:27:58.329380 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:27:58.329380 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:27:58.329380 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:27:58.329380 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:27:58.330611 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:27:58.330611 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:27:58.330611 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:27:58.330611 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:27:58.330611 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:27:58.330611 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:27:58.330611 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:27:58.330611 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:27:58.330611 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:27:58.330611 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:27:58.330611 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:27:58.330611 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:27:58.330611 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:27:58.330611 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:27:58.330611 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:27:58.330611 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:27:58.333650 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:27:58.333650 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:27:58.333650 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:27:58.333650 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:27:58.333650 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:27:58.333650 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:27:58.333650 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:27:58.333650 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:27:58.333650 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:27:58.333650 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:27:58.333650 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:27:58.333650 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:27:58.333650 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 610/1222] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 18 16:27:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:27:58.336976 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f6cd0e-2b58-4c03-bce9-734056c87af2 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:27:58.434524 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f6cd0e-2b58-4c03-bce9-734056c87af2 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:27:58.458669 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:27:58.471522 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70784b31-fc13-4b04-adb4-2dc255e3da87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:27:58.471522 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70784b31-fc13-4b04-adb4-2dc255e3da87 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:27:58.471522 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70784b31-fc13-4b04-adb4-2dc255e3da87 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:27:58.471522 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70784b31-fc13-4b04-adb4-2dc255e3da87 None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:27:58.471910 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70784b31-fc13-4b04-adb4-2dc255e3da87 None admin] RBAC: Adding query filter params (name=service) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:27:58.471910 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70784b31-fc13-4b04-adb4-2dc255e3da87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:27:58.473218 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70784b31-fc13-4b04-adb4-2dc255e3da87 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:27:58.481284 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 613/1223] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 18 16:27:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:27:58.488172 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa87fda4-970f-487e-b9c5-8c7b731cb843 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:27:58.531314 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:27:58.588146 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa87fda4-970f-487e-b9c5-8c7b731cb843 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:27:58.622186 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2980d6ae-a890-4432-95dc-9e786a393a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:27:58.623024 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980d6ae-a890-4432-95dc-9e786a393a75 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:27:58.623735 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980d6ae-a890-4432-95dc-9e786a393a75 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:27:58.623735 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980d6ae-a890-4432-95dc-9e786a393a75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:27:58.634227 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2980d6ae-a890-4432-95dc-9e786a393a75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:27:58.637663 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2980d6ae-a890-4432-95dc-9e786a393a75 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:27:58.652345 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-2980d6ae-a890-4432-95dc-9e786a393a75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:27:58.652345 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:27:58.652345 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:27:58.652345 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:27:58.652345 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:27:58.652345 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:27:58.652345 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:27:58.652345 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:27:58.652345 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:27:58.652345 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:27:58.652345 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:27:58.652345 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:27:58.652345 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:27:58.652345 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:27:58.652345 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:27:58.653426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:27:58.653426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:27:58.653426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:27:58.653426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:27:58.653426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:27:58.653426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:27:58.653426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:27:58.653426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:27:58.653426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:27:58.653426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:27:58.653426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:27:58.653426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:27:58.653426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:27:58.653426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:27:58.653426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:27:58.653426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:27:58.653426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:27:58.655416 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:27:58.655416 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:27:58.655416 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:27:58.655416 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:27:58.655416 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 611/1224] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:27:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:27:58.659990 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70784b31-fc13-4b04-adb4-2dc255e3da87 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:27:58.764372 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70784b31-fc13-4b04-adb4-2dc255e3da87 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:27:58.800331 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15807a17-0e15-445d-8893-a02d707df53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:27:58.800998 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15807a17-0e15-445d-8893-a02d707df53c None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:27:58.801299 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15807a17-0e15-445d-8893-a02d707df53c None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:27:58.801614 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15807a17-0e15-445d-8893-a02d707df53c None admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:27:58.802339 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15807a17-0e15-445d-8893-a02d707df53c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:27:58.802637 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15807a17-0e15-445d-8893-a02d707df53c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:27:58.804030 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15807a17-0e15-445d-8893-a02d707df53c None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:27:58.816101 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 614/1225] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:27:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:27:58.821906 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2980d6ae-a890-4432-95dc-9e786a393a75 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:27:58.929281 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:27:58.935742 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2980d6ae-a890-4432-95dc-9e786a393a75 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:27:58.970658 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8827ace8-f46e-4add-8076-9e56a5af3f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:27:58.971448 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827ace8-f46e-4add-8076-9e56a5af3f67 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:27:58.971764 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827ace8-f46e-4add-8076-9e56a5af3f67 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:27:58.972037 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827ace8-f46e-4add-8076-9e56a5af3f67 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:27:58.978956 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8827ace8-f46e-4add-8076-9e56a5af3f67 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:27:58.980510 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8827ace8-f46e-4add-8076-9e56a5af3f67 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:27:58.995176 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-8827ace8-f46e-4add-8076-9e56a5af3f67 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 16:27:58.995176 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:27:58.995176 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:27:58.995176 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:27:58.995176 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:27:58.995176 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:27:58.995176 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:27:58.995176 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:27:58.995176 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:27:58.995176 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:27:58.995176 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:27:58.995176 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:27:58.995176 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:27:58.995176 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:27:58.995176 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:27:58.995176 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:27:58.996197 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:27:58.996197 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:27:58.996197 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:27:58.996197 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:27:58.996197 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:27:58.996197 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:27:58.996197 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:27:58.996197 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:27:58.996197 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:27:58.996197 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:27:58.996197 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:27:58.996197 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:27:58.996197 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:27:58.996197 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:27:58.996197 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:27:58.996197 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:27:58.996197 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:27:58.996197 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:27:58.997350 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:27:58.997350 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:27:58.997350 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:27:58.997350 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:27:58.997350 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:27:58.997350 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:27:58.997350 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:27:58.997350 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:27:58.997350 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 16:27:58.997350 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:27:58.997350 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 612/1226] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:27:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:27:59.002565 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15807a17-0e15-445d-8893-a02d707df53c None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:27:59.050071 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:27:59.102173 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15807a17-0e15-445d-8893-a02d707df53c None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:27:59.135239 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f5237a-8b31-4893-b3df-abbf423d8cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:27:59.135937 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f5237a-8b31-4893-b3df-abbf423d8cb1 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:27:59.136238 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f5237a-8b31-4893-b3df-abbf423d8cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:27:59.136509 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f5237a-8b31-4893-b3df-abbf423d8cb1 None admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:27:59.137342 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f5237a-8b31-4893-b3df-abbf423d8cb1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:27:59.137647 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f5237a-8b31-4893-b3df-abbf423d8cb1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:27:59.140367 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f5237a-8b31-4893-b3df-abbf423d8cb1 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:27:59.168156 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 615/1227] 10.222.0.26 () {56 vars in 1143 bytes} [Mon Jul 18 16:27:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:27:59.174839 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8827ace8-f46e-4add-8076-9e56a5af3f67 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:27:59.277075 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8827ace8-f46e-4add-8076-9e56a5af3f67 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:27:59.312522 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c83d53-b9c8-4a3d-8c5b-d809b7f02896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:27:59.313237 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c83d53-b9c8-4a3d-8c5b-d809b7f02896 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:27:59.313518 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c83d53-b9c8-4a3d-8c5b-d809b7f02896 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:27:59.313791 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c83d53-b9c8-4a3d-8c5b-d809b7f02896 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:27:59.323738 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c83d53-b9c8-4a3d-8c5b-d809b7f02896 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:27:59.325178 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c83d53-b9c8-4a3d-8c5b-d809b7f02896 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:27:59.343511 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-38c83d53-b9c8-4a3d-8c5b-d809b7f02896 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:27:59.343511 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:27:59.343511 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:27:59.343511 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:27:59.343511 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:27:59.343511 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:27:59.343511 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:27:59.343511 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:27:59.343511 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:27:59.343511 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:27:59.343511 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:27:59.343511 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:27:59.343511 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:27:59.343511 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:27:59.343511 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:27:59.344624 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:27:59.344624 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:27:59.344624 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:27:59.344624 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:27:59.344624 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:27:59.344624 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:27:59.344624 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:27:59.344624 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:27:59.344624 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:27:59.344624 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:27:59.344624 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:27:59.344624 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:27:59.344624 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:27:59.344624 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:27:59.344624 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:27:59.344624 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:27:59.344624 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:27:59.345764 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:27:59.345764 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:27:59.345764 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:27:59.345764 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:27:59.347662 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 613/1228] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:27:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:27:59.350826 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f5237a-8b31-4893-b3df-abbf423d8cb1 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:27:59.453803 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f5237a-8b31-4893-b3df-abbf423d8cb1 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:27:59.456438 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:27:59.493433 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d32934-b663-4194-9fe0-7ba919766c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:27:59.494093 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d32934-b663-4194-9fe0-7ba919766c57 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:27:59.494373 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d32934-b663-4194-9fe0-7ba919766c57 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:27:59.494675 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d32934-b663-4194-9fe0-7ba919766c57 None admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:27:59.495405 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d32934-b663-4194-9fe0-7ba919766c57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:27:59.495736 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d32934-b663-4194-9fe0-7ba919766c57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:27:59.497146 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d32934-b663-4194-9fe0-7ba919766c57 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:27:59.508699 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 616/1229] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:27:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:27:59.515673 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c83d53-b9c8-4a3d-8c5b-d809b7f02896 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:27:59.524422 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:27:59.615399 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c83d53-b9c8-4a3d-8c5b-d809b7f02896 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:27:59.648504 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dbd51e-2b80-4696-a8ca-af4b3cf4dd6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:27:59.649335 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dbd51e-2b80-4696-a8ca-af4b3cf4dd6d None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:27:59.649611 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dbd51e-2b80-4696-a8ca-af4b3cf4dd6d None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:27:59.649882 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dbd51e-2b80-4696-a8ca-af4b3cf4dd6d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:27:59.657162 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dbd51e-2b80-4696-a8ca-af4b3cf4dd6d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:27:59.658592 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dbd51e-2b80-4696-a8ca-af4b3cf4dd6d None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:27:59.672886 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-22dbd51e-2b80-4696-a8ca-af4b3cf4dd6d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:27:59.672886 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:27:59.672886 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:27:59.672886 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:27:59.672886 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:27:59.672886 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:27:59.672886 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:27:59.672886 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:27:59.672886 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:27:59.672886 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:27:59.672886 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:27:59.672886 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:27:59.672886 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:27:59.672886 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:27:59.672886 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:27:59.674025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:27:59.674025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:27:59.674025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:27:59.674025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:27:59.674025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:27:59.674025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:27:59.674025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:27:59.674025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:27:59.674025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:27:59.674025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:27:59.674025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:27:59.674025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:27:59.674025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:27:59.674025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:27:59.674025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:27:59.674025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:27:59.674025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:27:59.679867 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:27:59.679867 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:27:59.679867 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:27:59.679867 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:27:59.679867 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:27:59.679867 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:27:59.679867 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:27:59.679867 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:27:59.679867 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 614/1230] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:27:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:27:59.680451 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d32934-b663-4194-9fe0-7ba919766c57 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:27:59.782158 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d32934-b663-4194-9fe0-7ba919766c57 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:27:59.818412 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a423c-1112-4457-978d-e27cce62f972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:27:59.819205 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a423c-1112-4457-978d-e27cce62f972 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:27:59.819516 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a423c-1112-4457-978d-e27cce62f972 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:27:59.819846 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a423c-1112-4457-978d-e27cce62f972 None admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:27:59.820627 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2a423c-1112-4457-978d-e27cce62f972 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:27:59.820934 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2a423c-1112-4457-978d-e27cce62f972 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:27:59.822358 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2a423c-1112-4457-978d-e27cce62f972 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:27:59.837904 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 617/1231] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 18 16:27:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:27:59.844870 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dbd51e-2b80-4696-a8ca-af4b3cf4dd6d None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:27:59.910709 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:27:59.945460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dbd51e-2b80-4696-a8ca-af4b3cf4dd6d None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:27:59.979043 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53baa75-c587-4d57-98ba-248926afc1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:27:59.979952 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53baa75-c587-4d57-98ba-248926afc1be None admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:27:59.980238 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53baa75-c587-4d57-98ba-248926afc1be None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:27:59.980517 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53baa75-c587-4d57-98ba-248926afc1be None admin] PATH_INFO: `/v3/projects/eb9747e2299c4e0991fd7edb4462c7e5/users/70f704a231a5474e8067a6c2dd88b355/roles/4fe970e98f394bbfb096c4879bebdf97` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:27:59.995033 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53baa75-c587-4d57-98ba-248926afc1be None admin] RBAC: Authorizing `identity:create_grant(project_id=eb9747e2299c4e0991fd7edb4462c7e5, user_id=70f704a231a5474e8067a6c2dd88b355, role_id=4fe970e98f394bbfb096c4879bebdf97)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:27:59.996506 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53baa75-c587-4d57-98ba-248926afc1be None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:00.130295 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 615/1232] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 18 16:27:59 2022] PUT /identity/v3/projects/eb9747e2299c4e0991fd7edb4462c7e5/users/70f704a231a5474e8067a6c2dd88b355/roles/4fe970e98f394bbfb096c4879bebdf97 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:28:02.159397 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd60aa-c602-4d78-b3d3-38d0c22a7ae7 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:02.159996 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd60aa-c602-4d78-b3d3-38d0c22a7ae7 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:02.159996 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd60aa-c602-4d78-b3d3-38d0c22a7ae7 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:02.161384 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 618/1233] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:28:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:28:02.166790 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f9052-1a49-4ca0-958e-f72f0797cb8f None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:02.167201 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f9052-1a49-4ca0-958e-f72f0797cb8f None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:02.167344 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f9052-1a49-4ca0-958e-f72f0797cb8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:02.184216 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-a86f9052-1a49-4ca0-958e-f72f0797cb8f None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:28:02.185934 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a86f9052-1a49-4ca0-958e-f72f0797cb8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:28:02.255921 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 616/1234] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:28:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:28:02.280045 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd60aa-c602-4d78-b3d3-38d0c22a7ae7 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:02.312145 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:02.397047 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd60aa-c602-4d78-b3d3-38d0c22a7ae7 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:02.433456 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc5b33-6f6b-457c-8636-a86e90fe879d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:02.434174 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc5b33-6f6b-457c-8636-a86e90fe879d None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:02.434454 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc5b33-6f6b-457c-8636-a86e90fe879d None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:02.434723 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc5b33-6f6b-457c-8636-a86e90fe879d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:02.449859 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc5b33-6f6b-457c-8636-a86e90fe879d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:28:02.459016 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-cddc5b33-6f6b-457c-8636-a86e90fe879d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:28:02.459016 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:02.459016 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:02.459016 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:02.459016 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:02.459016 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:02.459016 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:02.459016 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:02.459016 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:02.459016 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:02.459016 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:02.459016 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:02.459016 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:28:02.459016 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:28:02.459016 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:28:02.459016 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:28:02.460069 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:28:02.460069 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:28:02.460069 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:28:02.460069 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:28:02.460069 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:02.460069 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:02.460069 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:02.460069 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:02.460069 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:02.460069 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:02.460069 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:02.460069 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:02.460069 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:02.460069 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:02.460069 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:02.460069 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:02.461168 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:02.461168 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:02.461168 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:02.461168 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:02.461168 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:28:02.461168 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:28:02.461168 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:28:02.461168 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:28:02.461168 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:28:02.461168 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:28:02.461168 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:28:02.461168 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:28:02.461168 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 619/1235] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 18 16:28:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:28:02.466455 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86f9052-1a49-4ca0-958e-f72f0797cb8f None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:02.514370 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:02.567375 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86f9052-1a49-4ca0-958e-f72f0797cb8f None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:02.601829 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a92f61-f303-4cfb-b1dc-ee5343b3cd89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:02.602494 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a92f61-f303-4cfb-b1dc-ee5343b3cd89 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:02.602778 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a92f61-f303-4cfb-b1dc-ee5343b3cd89 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:02.603053 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a92f61-f303-4cfb-b1dc-ee5343b3cd89 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:02.603828 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a92f61-f303-4cfb-b1dc-ee5343b3cd89 None admin] RBAC: Adding query filter params (name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:02.604131 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a92f61-f303-4cfb-b1dc-ee5343b3cd89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:02.605497 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a92f61-f303-4cfb-b1dc-ee5343b3cd89 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:02.614074 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 617/1236] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 18 16:28:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:02.620057 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc5b33-6f6b-457c-8636-a86e90fe879d None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:02.717409 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:02.733251 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc5b33-6f6b-457c-8636-a86e90fe879d None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:02.768737 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c0b703-6794-4511-bebe-ecc598be3511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:02.769552 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0b703-6794-4511-bebe-ecc598be3511 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:02.769841 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0b703-6794-4511-bebe-ecc598be3511 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:02.770123 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0b703-6794-4511-bebe-ecc598be3511 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:02.781936 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c0b703-6794-4511-bebe-ecc598be3511 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:02.783432 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c0b703-6794-4511-bebe-ecc598be3511 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:02.802324 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-39c0b703-6794-4511-bebe-ecc598be3511 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:28:02.802324 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:02.802324 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:02.802324 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:02.802324 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:02.802324 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:02.802324 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:02.802324 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:02.802324 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:02.802324 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:02.802324 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:02.802324 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:02.802324 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:28:02.802324 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:28:02.802324 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:28:02.809200 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:28:02.809200 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:02.809200 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:02.809200 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:02.809200 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:02.809200 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:02.809200 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:02.809200 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:02.809200 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:02.809200 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:02.809200 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:02.809200 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:02.809200 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:02.809200 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:02.809200 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:02.809200 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:02.809200 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:02.810773 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:28:02.810773 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:28:02.810773 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:28:02.810773 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:28:02.810773 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 620/1237] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:28:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:28:02.811922 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a92f61-f303-4cfb-b1dc-ee5343b3cd89 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:02.926852 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a92f61-f303-4cfb-b1dc-ee5343b3cd89 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:02.936160 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:02.961239 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5ed744-8ea0-4b8f-a176-dd708b1c4003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:02.961902 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ed744-8ea0-4b8f-a176-dd708b1c4003 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:02.962188 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ed744-8ea0-4b8f-a176-dd708b1c4003 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:02.962469 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ed744-8ea0-4b8f-a176-dd708b1c4003 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:02.963262 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5ed744-8ea0-4b8f-a176-dd708b1c4003 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:02.963601 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5ed744-8ea0-4b8f-a176-dd708b1c4003 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:02.964977 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5ed744-8ea0-4b8f-a176-dd708b1c4003 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:02.977180 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 618/1238] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:28:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:02.983347 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c0b703-6794-4511-bebe-ecc598be3511 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:03.087941 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c0b703-6794-4511-bebe-ecc598be3511 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:03.122469 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec11674-b811-4d59-86b9-c0ecfa080b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:03.123832 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec11674-b811-4d59-86b9-c0ecfa080b61 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:03.124113 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec11674-b811-4d59-86b9-c0ecfa080b61 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:03.124386 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec11674-b811-4d59-86b9-c0ecfa080b61 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:03.127163 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:03.131989 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec11674-b811-4d59-86b9-c0ecfa080b61 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:03.133511 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec11674-b811-4d59-86b9-c0ecfa080b61 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:03.149170 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-eec11674-b811-4d59-86b9-c0ecfa080b61 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 16:28:03.149170 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:03.149170 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:03.149170 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:03.149170 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:03.149170 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:03.149170 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:03.149170 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:03.149170 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:03.149170 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:03.149170 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:03.149170 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:03.149170 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:28:03.149170 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:28:03.149170 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:28:03.149170 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:28:03.158489 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:03.158489 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:03.158489 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:28:03.158489 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:28:03.158489 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:28:03.158489 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:28:03.158489 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:03.158489 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:03.158489 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:03.158489 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:03.158489 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:03.158489 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:03.158489 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:03.158489 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:03.158489 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:03.158489 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:03.158489 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:03.158489 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:03.159960 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:03.159960 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:03.159960 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:28:03.159960 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:28:03.159960 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:28:03.159960 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:28:03.159960 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:28:03.159960 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:28:03.159960 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 16:28:03.159960 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:28:03.159960 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 621/1239] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:28:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:28:03.160560 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5ed744-8ea0-4b8f-a176-dd708b1c4003 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:03.265176 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5ed744-8ea0-4b8f-a176-dd708b1c4003 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:03.300083 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c883abff-26de-4252-a90c-f2e44fd7080b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:03.300759 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883abff-26de-4252-a90c-f2e44fd7080b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:03.301041 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883abff-26de-4252-a90c-f2e44fd7080b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:03.301316 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883abff-26de-4252-a90c-f2e44fd7080b None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:03.302149 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c883abff-26de-4252-a90c-f2e44fd7080b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:03.302458 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c883abff-26de-4252-a90c-f2e44fd7080b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:03.303920 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c883abff-26de-4252-a90c-f2e44fd7080b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:03.322704 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 619/1240] 10.222.0.26 () {56 vars in 1143 bytes} [Mon Jul 18 16:28:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:03.329206 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec11674-b811-4d59-86b9-c0ecfa080b61 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:03.440980 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec11674-b811-4d59-86b9-c0ecfa080b61 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:03.476063 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e031bb-c11c-434a-8287-2409f4e6ab0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:03.476721 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e031bb-c11c-434a-8287-2409f4e6ab0d None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:03.477002 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e031bb-c11c-434a-8287-2409f4e6ab0d None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:03.477275 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e031bb-c11c-434a-8287-2409f4e6ab0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:03.542889 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:03.550484 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e031bb-c11c-434a-8287-2409f4e6ab0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:03.552029 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e031bb-c11c-434a-8287-2409f4e6ab0d None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:03.654391 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 622/1241] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:28:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 16:28:03.660406 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c883abff-26de-4252-a90c-f2e44fd7080b None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:03.688781 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:03.764844 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c883abff-26de-4252-a90c-f2e44fd7080b None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:03.804708 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9272646-1418-46fd-9d48-a59e847cee55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:03.805621 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9272646-1418-46fd-9d48-a59e847cee55 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:03.805995 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9272646-1418-46fd-9d48-a59e847cee55 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:03.806399 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9272646-1418-46fd-9d48-a59e847cee55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:03.817869 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9272646-1418-46fd-9d48-a59e847cee55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:03.819395 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9272646-1418-46fd-9d48-a59e847cee55 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:03.838038 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-f9272646-1418-46fd-9d48-a59e847cee55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:28:03.838038 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:03.838038 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:03.838038 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:03.838038 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:03.838038 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:03.838038 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:03.838038 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:03.838038 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:03.838038 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:03.838038 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:03.838038 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:03.838038 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:28:03.838038 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:28:03.838038 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:28:03.839121 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:28:03.839121 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:03.839121 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:03.839121 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:03.839121 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:03.839121 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:03.839121 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:03.839121 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:03.839121 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:03.839121 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:03.839121 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:03.839121 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:03.839121 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:03.839121 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:03.839121 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:03.839121 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:03.839121 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:03.840063 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:28:03.840063 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:28:03.840063 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:28:03.840063 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:28:03.840063 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 620/1242] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:28:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:28:03.845885 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e031bb-c11c-434a-8287-2409f4e6ab0d None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:03.943948 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:03.949661 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e031bb-c11c-434a-8287-2409f4e6ab0d None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:03.984262 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dcb63c-55e4-45ee-afbd-47be1cbb5921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:03.985260 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcb63c-55e4-45ee-afbd-47be1cbb5921 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:03.985260 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcb63c-55e4-45ee-afbd-47be1cbb5921 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:03.985506 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcb63c-55e4-45ee-afbd-47be1cbb5921 None admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:03.986198 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dcb63c-55e4-45ee-afbd-47be1cbb5921 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:03.986500 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dcb63c-55e4-45ee-afbd-47be1cbb5921 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:03.987896 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dcb63c-55e4-45ee-afbd-47be1cbb5921 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:04.000084 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 623/1243] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:28:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:04.005929 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9272646-1418-46fd-9d48-a59e847cee55 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:04.086198 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:04.108202 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9272646-1418-46fd-9d48-a59e847cee55 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:04.143650 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a606541-411b-40b6-953f-35a4deb6ffa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:04.144444 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a606541-411b-40b6-953f-35a4deb6ffa9 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:04.144728 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a606541-411b-40b6-953f-35a4deb6ffa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:04.145000 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a606541-411b-40b6-953f-35a4deb6ffa9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:04.151805 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a606541-411b-40b6-953f-35a4deb6ffa9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:04.153240 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a606541-411b-40b6-953f-35a4deb6ffa9 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:04.167039 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-4a606541-411b-40b6-953f-35a4deb6ffa9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:28:04.167039 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:04.167039 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:04.167039 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:04.167039 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:04.167039 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:04.167039 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:04.167039 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:04.167039 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:04.167039 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:04.167039 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:04.167039 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:04.167039 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:28:04.167039 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:28:04.167039 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:28:04.168172 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:28:04.168172 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:04.168172 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:04.168172 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:04.168172 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:04.168172 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:04.168172 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:04.168172 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:04.168172 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:04.168172 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:04.168172 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:04.168172 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:04.168172 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:04.168172 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:04.168172 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:04.168172 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:04.168172 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:04.169221 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:28:04.169221 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:28:04.169221 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:28:04.169221 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:28:04.169221 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:28:04.169221 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:28:04.169221 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:28:04.169221 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:28:04.169221 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 621/1244] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:28:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:28:04.174845 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dcb63c-55e4-45ee-afbd-47be1cbb5921 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:04.277854 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dcb63c-55e4-45ee-afbd-47be1cbb5921 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:04.312977 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58902119-bc8d-4dc0-803d-d60e769cb256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:04.313633 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58902119-bc8d-4dc0-803d-d60e769cb256 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:04.313913 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58902119-bc8d-4dc0-803d-d60e769cb256 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:04.314183 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58902119-bc8d-4dc0-803d-d60e769cb256 None admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:04.314984 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58902119-bc8d-4dc0-803d-d60e769cb256 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:04.315282 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58902119-bc8d-4dc0-803d-d60e769cb256 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:04.316734 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58902119-bc8d-4dc0-803d-d60e769cb256 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:04.329472 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 624/1245] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 18 16:28:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:04.335931 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a606541-411b-40b6-953f-35a4deb6ffa9 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:04.444762 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a606541-411b-40b6-953f-35a4deb6ffa9 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:04.483373 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef76078-e466-4b7a-94db-f7a0f727d837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:04.484242 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef76078-e466-4b7a-94db-f7a0f727d837 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:04.484523 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef76078-e466-4b7a-94db-f7a0f727d837 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:04.484794 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef76078-e466-4b7a-94db-f7a0f727d837 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:04.485769 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef76078-e466-4b7a-94db-f7a0f727d837 None admin] RBAC: Adding query filter params (role.id=4fe970e98f394bbfb096c4879bebdf97, scope.project.id=eb9747e2299c4e0991fd7edb4462c7e5, user.id=70f704a231a5474e8067a6c2dd88b355) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:04.486083 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef76078-e466-4b7a-94db-f7a0f727d837 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:04.488461 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef76078-e466-4b7a-94db-f7a0f727d837 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:04.497241 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 622/1246] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 18 16:28:04 2022] GET /identity/v3/role_assignments?user.id=70f704a231a5474e8067a6c2dd88b355&scope.project.id=eb9747e2299c4e0991fd7edb4462c7e5&role.id=4fe970e98f394bbfb096c4879bebdf97 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:06.714976 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e623f6b7-ce36-4447-be3e-e650c157e1a6 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:06.715805 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e623f6b7-ce36-4447-be3e-e650c157e1a6 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:06.715805 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e623f6b7-ce36-4447-be3e-e650c157e1a6 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:06.719313 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 625/1247] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:28:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:28:06.723848 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397f3f4-7db8-448b-8c3c-d54862260d2d None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:06.724336 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397f3f4-7db8-448b-8c3c-d54862260d2d None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:06.724478 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397f3f4-7db8-448b-8c3c-d54862260d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:06.729846 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:06.742756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-d397f3f4-7db8-448b-8c3c-d54862260d2d None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:28:06.744105 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d397f3f4-7db8-448b-8c3c-d54862260d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:28:06.817367 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 623/1248] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:28:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:28:06.842631 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e623f6b7-ce36-4447-be3e-e650c157e1a6 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:06.862627 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:06.951272 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e623f6b7-ce36-4447-be3e-e650c157e1a6 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:06.987530 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d83665-384c-46a2-a278-74baacf73f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:06.988271 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d83665-384c-46a2-a278-74baacf73f25 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:06.988557 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d83665-384c-46a2-a278-74baacf73f25 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:06.988838 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d83665-384c-46a2-a278-74baacf73f25 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:06.999061 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d83665-384c-46a2-a278-74baacf73f25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:28:07.006927 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-58d83665-384c-46a2-a278-74baacf73f25 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:28:07.006927 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:07.006927 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:07.006927 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:07.006927 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:07.006927 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:07.006927 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:07.006927 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:07.006927 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:07.006927 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:07.006927 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:07.006927 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:07.006927 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:28:07.006927 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:28:07.006927 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:28:07.006927 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:28:07.013462 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:28:07.013462 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:28:07.013462 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:28:07.013462 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:28:07.013462 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:07.013462 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:07.013462 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:07.013462 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:07.013462 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:07.013462 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:07.013462 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:07.013462 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:07.013462 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:07.013462 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:07.013462 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:07.013462 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:07.014880 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:07.014880 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:07.014880 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:07.014880 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:07.014880 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:28:07.014880 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:28:07.014880 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:28:07.014880 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:28:07.014880 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:28:07.014880 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:28:07.014880 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:28:07.014880 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:28:07.014880 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 626/1249] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 18 16:28:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:28:07.015887 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d397f3f4-7db8-448b-8c3c-d54862260d2d None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:07.116105 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d397f3f4-7db8-448b-8c3c-d54862260d2d None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:07.145978 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:07.150756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb67545a-570f-4aeb-ad10-3669abd0f2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:07.151428 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67545a-570f-4aeb-ad10-3669abd0f2dc None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:07.151745 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67545a-570f-4aeb-ad10-3669abd0f2dc None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:07.152027 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67545a-570f-4aeb-ad10-3669abd0f2dc None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:07.152769 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb67545a-570f-4aeb-ad10-3669abd0f2dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:07.153078 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb67545a-570f-4aeb-ad10-3669abd0f2dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:07.154779 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb67545a-570f-4aeb-ad10-3669abd0f2dc None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:07.162983 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 624/1250] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:28:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:07.169371 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d83665-384c-46a2-a278-74baacf73f25 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:07.250744 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:07.275889 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d83665-384c-46a2-a278-74baacf73f25 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:07.310688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ebea01-181d-4204-a953-78d5226c9057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:07.311529 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ebea01-181d-4204-a953-78d5226c9057 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:07.311817 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ebea01-181d-4204-a953-78d5226c9057 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:07.312217 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ebea01-181d-4204-a953-78d5226c9057 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:07.323137 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ebea01-181d-4204-a953-78d5226c9057 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:07.324853 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ebea01-181d-4204-a953-78d5226c9057 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:07.343654 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-60ebea01-181d-4204-a953-78d5226c9057 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:28:07.343654 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:07.343654 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:07.343654 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:07.343654 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:07.343654 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:07.343654 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:07.343654 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:07.343654 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:07.343654 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:07.343654 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:07.343654 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:07.343654 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:28:07.343654 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:28:07.343654 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:28:07.344768 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:28:07.344768 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:07.344768 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:07.344768 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:07.344768 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:07.344768 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:07.344768 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:07.344768 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:07.344768 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:07.344768 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:07.344768 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:07.344768 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:07.344768 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:07.344768 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:07.344768 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:07.344768 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:07.344768 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:07.346026 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:28:07.346026 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:28:07.346026 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:28:07.346026 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:28:07.346026 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 627/1251] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:28:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:28:07.351151 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb67545a-570f-4aeb-ad10-3669abd0f2dc None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:07.452830 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb67545a-570f-4aeb-ad10-3669abd0f2dc None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:07.486859 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b640a9-fbc4-4003-9ff8-ceb929b7c1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:07.487594 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b640a9-fbc4-4003-9ff8-ceb929b7c1d0 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:07.487947 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b640a9-fbc4-4003-9ff8-ceb929b7c1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:07.488225 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b640a9-fbc4-4003-9ff8-ceb929b7c1d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:07.488940 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b640a9-fbc4-4003-9ff8-ceb929b7c1d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:07.489242 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b640a9-fbc4-4003-9ff8-ceb929b7c1d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:07.490611 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b640a9-fbc4-4003-9ff8-ceb929b7c1d0 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:07.503788 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 625/1252] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:28:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:07.509522 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ebea01-181d-4204-a953-78d5226c9057 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:07.612579 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ebea01-181d-4204-a953-78d5226c9057 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:07.640350 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:07.649666 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8bd353-66ff-4d02-957f-9a0d00c65e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:07.651281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8bd353-66ff-4d02-957f-9a0d00c65e9d None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:07.651604 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8bd353-66ff-4d02-957f-9a0d00c65e9d None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:07.651884 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8bd353-66ff-4d02-957f-9a0d00c65e9d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:07.666004 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8bd353-66ff-4d02-957f-9a0d00c65e9d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:07.667456 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8bd353-66ff-4d02-957f-9a0d00c65e9d None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:07.682934 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-9a8bd353-66ff-4d02-957f-9a0d00c65e9d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 16:28:07.682934 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:07.682934 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:07.682934 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:07.682934 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:07.682934 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:07.682934 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:07.682934 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:07.682934 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:07.682934 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:07.682934 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:07.682934 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:07.682934 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:28:07.682934 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:28:07.682934 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:28:07.682934 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:28:07.684125 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:07.684125 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:07.684125 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:28:07.684125 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:28:07.684125 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:28:07.684125 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:28:07.684125 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:07.684125 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:07.684125 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:07.684125 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:07.684125 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:07.684125 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:07.684125 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:07.684125 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:07.684125 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:07.684125 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:07.684125 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:07.684125 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:07.685254 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:07.685254 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:07.685254 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:28:07.685254 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:28:07.685254 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:28:07.685254 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:28:07.685254 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:28:07.685254 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:28:07.685254 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 16:28:07.685254 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:28:07.685254 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 628/1253] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:28:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:28:07.690458 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b640a9-fbc4-4003-9ff8-ceb929b7c1d0 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:07.748259 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:07.793809 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b640a9-fbc4-4003-9ff8-ceb929b7c1d0 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:07.827443 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef27a56e-f943-4e30-ba4d-8e2e53815c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:07.828358 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27a56e-f943-4e30-ba4d-8e2e53815c7f None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:07.828645 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27a56e-f943-4e30-ba4d-8e2e53815c7f None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:07.828919 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27a56e-f943-4e30-ba4d-8e2e53815c7f None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:07.829764 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef27a56e-f943-4e30-ba4d-8e2e53815c7f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:07.830072 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef27a56e-f943-4e30-ba4d-8e2e53815c7f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:07.831608 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef27a56e-f943-4e30-ba4d-8e2e53815c7f None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:07.850555 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 626/1254] 10.222.0.26 () {56 vars in 1143 bytes} [Mon Jul 18 16:28:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:07.856113 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8bd353-66ff-4d02-957f-9a0d00c65e9d None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:07.959715 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8bd353-66ff-4d02-957f-9a0d00c65e9d None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:07.994077 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420c6011-d739-48e6-bc7b-0eec8d4252c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:07.994743 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c6011-d739-48e6-bc7b-0eec8d4252c6 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:07.995027 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c6011-d739-48e6-bc7b-0eec8d4252c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:07.995305 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c6011-d739-48e6-bc7b-0eec8d4252c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:08.054819 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:08.068484 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420c6011-d739-48e6-bc7b-0eec8d4252c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:08.074639 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420c6011-d739-48e6-bc7b-0eec8d4252c6 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:08.168869 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 629/1255] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:28:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 16:28:08.176303 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef27a56e-f943-4e30-ba4d-8e2e53815c7f None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:08.286229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef27a56e-f943-4e30-ba4d-8e2e53815c7f None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:08.300649 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:08.322195 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed7a085-2276-4e95-8731-a7c7004804a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:08.322915 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7a085-2276-4e95-8731-a7c7004804a6 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:08.323200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7a085-2276-4e95-8731-a7c7004804a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:08.323542 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7a085-2276-4e95-8731-a7c7004804a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:08.333507 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed7a085-2276-4e95-8731-a7c7004804a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:08.334988 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed7a085-2276-4e95-8731-a7c7004804a6 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:08.355568 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-8ed7a085-2276-4e95-8731-a7c7004804a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:28:08.355568 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:08.355568 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:08.355568 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:08.355568 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:08.355568 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:08.355568 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:08.355568 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:08.355568 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:08.355568 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:08.355568 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:08.355568 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:08.355568 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:28:08.355568 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:28:08.355568 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:28:08.356823 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:28:08.356823 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:08.356823 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:08.356823 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:08.356823 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:08.356823 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:08.356823 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:08.356823 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:08.356823 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:08.356823 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:08.356823 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:08.356823 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:08.356823 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:08.356823 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:08.356823 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:08.356823 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:08.356823 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:08.357803 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:28:08.357803 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:28:08.357803 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:28:08.357803 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:28:08.357803 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 627/1256] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:28:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:28:08.363895 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420c6011-d739-48e6-bc7b-0eec8d4252c6 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:08.453901 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:08.465574 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420c6011-d739-48e6-bc7b-0eec8d4252c6 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:08.501276 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d627ff-208d-432f-8d4b-1267e9a9089f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:08.502001 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d627ff-208d-432f-8d4b-1267e9a9089f None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:08.502324 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d627ff-208d-432f-8d4b-1267e9a9089f None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:08.502643 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d627ff-208d-432f-8d4b-1267e9a9089f None admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:08.503432 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d627ff-208d-432f-8d4b-1267e9a9089f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:08.503835 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d627ff-208d-432f-8d4b-1267e9a9089f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:08.505358 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d627ff-208d-432f-8d4b-1267e9a9089f None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:08.518702 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 630/1257] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:28:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:08.524513 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed7a085-2276-4e95-8731-a7c7004804a6 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:08.630388 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed7a085-2276-4e95-8731-a7c7004804a6 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:08.673466 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea5e94f-2e65-41a8-99b1-af8be494cad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:08.674298 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5e94f-2e65-41a8-99b1-af8be494cad8 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:08.674583 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5e94f-2e65-41a8-99b1-af8be494cad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:08.674858 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5e94f-2e65-41a8-99b1-af8be494cad8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:08.683062 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea5e94f-2e65-41a8-99b1-af8be494cad8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:08.684626 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea5e94f-2e65-41a8-99b1-af8be494cad8 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:08.700657 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-2ea5e94f-2e65-41a8-99b1-af8be494cad8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:28:08.700657 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:08.700657 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:08.700657 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:08.700657 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:08.700657 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:08.700657 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:08.700657 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:08.700657 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:08.700657 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:08.700657 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:08.700657 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:08.700657 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:28:08.700657 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:28:08.700657 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:28:08.702460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:28:08.702460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:08.702460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:08.702460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:08.702460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:08.702460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:08.702460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:08.702460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:08.702460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:08.702460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:08.702460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:08.702460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:08.702460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:08.702460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:08.702460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:08.702460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:08.702460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:08.703443 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:28:08.703443 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:28:08.703443 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:28:08.703443 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:28:08.703443 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:28:08.703443 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:28:08.703443 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:28:08.703443 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:28:08.703443 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 628/1258] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:28:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:28:08.708969 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d627ff-208d-432f-8d4b-1267e9a9089f None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:08.816762 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d627ff-208d-432f-8d4b-1267e9a9089f None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:08.852797 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec5f724-9e77-4ef9-8e7d-aedbd14fbbe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:08.853468 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5f724-9e77-4ef9-8e7d-aedbd14fbbe8 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:08.853754 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5f724-9e77-4ef9-8e7d-aedbd14fbbe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:08.865594 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5f724-9e77-4ef9-8e7d-aedbd14fbbe8 None admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:08.865594 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec5f724-9e77-4ef9-8e7d-aedbd14fbbe8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:08.865594 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec5f724-9e77-4ef9-8e7d-aedbd14fbbe8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:08.865594 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec5f724-9e77-4ef9-8e7d-aedbd14fbbe8 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:08.869791 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 631/1259] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 18 16:28:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:08.877022 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea5e94f-2e65-41a8-99b1-af8be494cad8 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:08.903987 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:08.988438 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea5e94f-2e65-41a8-99b1-af8be494cad8 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:09.024006 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dc5c19-267d-43a6-9ba8-4355908982d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:09.024654 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc5c19-267d-43a6-9ba8-4355908982d9 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:09.024941 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc5c19-267d-43a6-9ba8-4355908982d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:09.025213 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc5c19-267d-43a6-9ba8-4355908982d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:09.026031 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dc5c19-267d-43a6-9ba8-4355908982d9 None admin] RBAC: Adding query filter params (role.id=98eaadb3669f4b608d572c001742f559, scope.project.id=eb9747e2299c4e0991fd7edb4462c7e5, user.id=70f704a231a5474e8067a6c2dd88b355) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:09.026406 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dc5c19-267d-43a6-9ba8-4355908982d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:09.027928 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dc5c19-267d-43a6-9ba8-4355908982d9 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:09.033521 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 629/1260] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 18 16:28:08 2022] GET /identity/v3/role_assignments?user.id=70f704a231a5474e8067a6c2dd88b355&scope.project.id=eb9747e2299c4e0991fd7edb4462c7e5&role.id=98eaadb3669f4b608d572c001742f559 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:11.110768 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e1ecc-3a09-4181-ab99-6ea669bbcfcb None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:11.111270 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e1ecc-3a09-4181-ab99-6ea669bbcfcb None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:11.111270 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e1ecc-3a09-4181-ab99-6ea669bbcfcb None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:11.112629 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 632/1261] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:28:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:28:11.117859 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e40636-61b8-410b-b76e-37e0149e1411 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:11.118265 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e40636-61b8-410b-b76e-37e0149e1411 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:11.118421 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e40636-61b8-410b-b76e-37e0149e1411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:11.135650 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-71e40636-61b8-410b-b76e-37e0149e1411 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:28:11.136915 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-71e40636-61b8-410b-b76e-37e0149e1411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:28:11.204659 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 630/1262] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:28:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:28:11.227251 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e1ecc-3a09-4181-ab99-6ea669bbcfcb None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:11.239891 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:11.341051 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e1ecc-3a09-4181-ab99-6ea669bbcfcb None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:11.375372 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a41a9-e5bc-42ae-8b69-0093c6ce3d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:11.376132 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a41a9-e5bc-42ae-8b69-0093c6ce3d97 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:11.376417 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a41a9-e5bc-42ae-8b69-0093c6ce3d97 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:11.376696 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a41a9-e5bc-42ae-8b69-0093c6ce3d97 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:11.387097 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392a41a9-e5bc-42ae-8b69-0093c6ce3d97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:28:11.395662 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-392a41a9-e5bc-42ae-8b69-0093c6ce3d97 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:28:11.395662 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:11.395662 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:11.395662 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:11.395662 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:11.395662 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:11.395662 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:11.395662 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:11.395662 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:11.395662 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:11.395662 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:11.395662 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:11.395662 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:28:11.395662 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:28:11.395662 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:28:11.395662 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:28:11.396805 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:28:11.396805 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:28:11.396805 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:28:11.396805 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:28:11.396805 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:11.396805 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:11.396805 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:11.396805 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:11.396805 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:11.396805 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:11.396805 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:11.396805 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:11.396805 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:11.396805 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:11.396805 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:11.396805 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:11.397830 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:11.397830 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:11.397830 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:11.397830 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:11.397830 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:28:11.397830 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:28:11.397830 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:28:11.397830 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:28:11.397830 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:28:11.397830 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:28:11.397830 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:28:11.397830 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:28:11.397830 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 633/1263] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 18 16:28:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:28:11.402987 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e40636-61b8-410b-b76e-37e0149e1411 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:11.422154 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:11.501792 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e40636-61b8-410b-b76e-37e0149e1411 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:11.537656 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb624bcd-81b7-4111-9e8c-d99513eb1200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:11.538324 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb624bcd-81b7-4111-9e8c-d99513eb1200 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:11.538607 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb624bcd-81b7-4111-9e8c-d99513eb1200 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:11.538880 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb624bcd-81b7-4111-9e8c-d99513eb1200 None admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:11.539905 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb624bcd-81b7-4111-9e8c-d99513eb1200 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:11.540211 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb624bcd-81b7-4111-9e8c-d99513eb1200 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:11.541571 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb624bcd-81b7-4111-9e8c-d99513eb1200 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:11.550425 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 631/1264] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:28:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:11.556304 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a41a9-e5bc-42ae-8b69-0093c6ce3d97 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:11.624419 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:11.657213 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a41a9-e5bc-42ae-8b69-0093c6ce3d97 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:11.690670 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a81e419-b196-471d-845a-5e0ef5181f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:11.691459 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81e419-b196-471d-845a-5e0ef5181f29 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:11.691777 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81e419-b196-471d-845a-5e0ef5181f29 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:11.692056 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81e419-b196-471d-845a-5e0ef5181f29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:11.702705 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a81e419-b196-471d-845a-5e0ef5181f29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:11.704199 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a81e419-b196-471d-845a-5e0ef5181f29 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:11.721645 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-1a81e419-b196-471d-845a-5e0ef5181f29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:28:11.721645 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:11.721645 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:11.721645 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:11.721645 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:11.721645 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:11.721645 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:11.721645 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:11.721645 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:11.721645 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:11.721645 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:11.721645 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:11.721645 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:28:11.721645 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:28:11.721645 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:28:11.722795 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:28:11.722795 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:11.722795 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:11.722795 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:11.722795 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:11.722795 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:11.722795 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:11.722795 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:11.722795 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:11.722795 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:11.722795 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:11.722795 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:11.722795 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:11.722795 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:11.722795 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:11.722795 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:11.722795 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:11.723956 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:28:11.723956 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:28:11.723956 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:28:11.723956 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:28:11.723956 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 634/1265] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:28:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:28:11.728996 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb624bcd-81b7-4111-9e8c-d99513eb1200 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:11.815507 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:11.835042 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb624bcd-81b7-4111-9e8c-d99513eb1200 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:11.868589 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac24c0da-3790-4122-9f68-86daefe6e157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:11.869251 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac24c0da-3790-4122-9f68-86daefe6e157 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:11.869536 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac24c0da-3790-4122-9f68-86daefe6e157 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:11.869814 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac24c0da-3790-4122-9f68-86daefe6e157 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:11.870537 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac24c0da-3790-4122-9f68-86daefe6e157 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:11.870843 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac24c0da-3790-4122-9f68-86daefe6e157 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:11.872239 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac24c0da-3790-4122-9f68-86daefe6e157 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:11.883623 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 632/1266] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:28:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:11.890010 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a81e419-b196-471d-845a-5e0ef5181f29 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:11.989445 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a81e419-b196-471d-845a-5e0ef5181f29 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:12.008896 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:12.022592 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6986d39d-3d5f-4933-a3ed-dc576f49597b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:12.023399 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6986d39d-3d5f-4933-a3ed-dc576f49597b None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:12.023711 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6986d39d-3d5f-4933-a3ed-dc576f49597b None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:12.023988 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6986d39d-3d5f-4933-a3ed-dc576f49597b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:12.032064 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6986d39d-3d5f-4933-a3ed-dc576f49597b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:12.033491 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6986d39d-3d5f-4933-a3ed-dc576f49597b None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:12.048488 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-6986d39d-3d5f-4933-a3ed-dc576f49597b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 16:28:12.048488 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:12.048488 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:12.048488 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:12.048488 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:12.048488 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:12.048488 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:12.048488 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:12.048488 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:12.048488 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:12.048488 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:12.048488 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:12.048488 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:28:12.048488 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:28:12.048488 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:28:12.048488 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:28:12.050334 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:12.050334 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:12.050334 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:28:12.050334 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:28:12.050334 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:28:12.050334 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:28:12.050334 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:12.050334 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:12.050334 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:12.050334 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:12.050334 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:12.050334 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:12.050334 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:12.050334 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:12.050334 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:12.050334 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:12.050334 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:12.050334 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:12.052664 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:12.052664 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:12.052664 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:28:12.052664 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:28:12.052664 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:28:12.052664 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:28:12.052664 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:28:12.052664 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:28:12.052664 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 16:28:12.052664 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:28:12.052664 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 635/1267] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:28:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:28:12.057343 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac24c0da-3790-4122-9f68-86daefe6e157 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:12.157235 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac24c0da-3790-4122-9f68-86daefe6e157 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:12.194085 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b06406-7855-46f8-bba0-bfa8e7b665d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:12.194760 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b06406-7855-46f8-bba0-bfa8e7b665d3 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:12.195055 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b06406-7855-46f8-bba0-bfa8e7b665d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:12.195306 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b06406-7855-46f8-bba0-bfa8e7b665d3 None admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:12.196249 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b06406-7855-46f8-bba0-bfa8e7b665d3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:12.196559 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b06406-7855-46f8-bba0-bfa8e7b665d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:12.197999 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b06406-7855-46f8-bba0-bfa8e7b665d3 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:12.207743 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:12.215887 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 633/1268] 10.222.0.26 () {56 vars in 1143 bytes} [Mon Jul 18 16:28:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:12.223036 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6986d39d-3d5f-4933-a3ed-dc576f49597b None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:12.323652 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6986d39d-3d5f-4933-a3ed-dc576f49597b None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:12.357397 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e73ed4-1bae-4614-a7b3-5b63e09c5314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:12.358187 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e73ed4-1bae-4614-a7b3-5b63e09c5314 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:12.358471 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e73ed4-1bae-4614-a7b3-5b63e09c5314 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:12.358747 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e73ed4-1bae-4614-a7b3-5b63e09c5314 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:12.369120 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e73ed4-1bae-4614-a7b3-5b63e09c5314 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:12.370560 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e73ed4-1bae-4614-a7b3-5b63e09c5314 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:12.388074 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-30e73ed4-1bae-4614-a7b3-5b63e09c5314 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:28:12.388074 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:12.388074 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:12.388074 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:12.388074 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:12.388074 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:12.388074 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:12.388074 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:12.388074 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:12.388074 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:12.388074 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:12.388074 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:12.388074 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:28:12.388074 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:28:12.388074 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:28:12.389689 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:28:12.389689 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:12.389689 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:12.389689 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:12.389689 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:12.389689 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:12.389689 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:12.389689 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:12.389689 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:12.389689 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:12.389689 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:12.389689 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:12.389689 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:12.389689 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:12.389689 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:12.389689 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:12.389689 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:12.390703 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:28:12.390703 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:28:12.390703 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:28:12.390703 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:28:12.390703 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 636/1269] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:28:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:28:12.395999 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b06406-7855-46f8-bba0-bfa8e7b665d3 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:12.494504 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b06406-7855-46f8-bba0-bfa8e7b665d3 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:12.528148 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566c931a-7b6f-47e6-a5b1-9e2e4ae4b03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:12.528801 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c931a-7b6f-47e6-a5b1-9e2e4ae4b03a None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:12.529098 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c931a-7b6f-47e6-a5b1-9e2e4ae4b03a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:12.529498 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c931a-7b6f-47e6-a5b1-9e2e4ae4b03a None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:12.530395 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566c931a-7b6f-47e6-a5b1-9e2e4ae4b03a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:12.530699 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566c931a-7b6f-47e6-a5b1-9e2e4ae4b03a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:12.532084 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566c931a-7b6f-47e6-a5b1-9e2e4ae4b03a None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:12.543589 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 634/1270] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:28:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:12.550118 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e73ed4-1bae-4614-a7b3-5b63e09c5314 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:12.582648 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:12.650403 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e73ed4-1bae-4614-a7b3-5b63e09c5314 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:12.685297 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c53c48-94a0-4689-8482-b2d3dae9c861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:12.686208 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c53c48-94a0-4689-8482-b2d3dae9c861 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:12.686499 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c53c48-94a0-4689-8482-b2d3dae9c861 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:12.686789 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c53c48-94a0-4689-8482-b2d3dae9c861 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:12.693322 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c53c48-94a0-4689-8482-b2d3dae9c861 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:12.694769 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c53c48-94a0-4689-8482-b2d3dae9c861 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:12.708868 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-71c53c48-94a0-4689-8482-b2d3dae9c861 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:28:12.708868 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:12.708868 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:12.708868 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:12.708868 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:12.708868 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:12.708868 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:12.708868 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:12.708868 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:12.708868 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:12.708868 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:12.708868 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:12.708868 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:28:12.708868 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:28:12.708868 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:28:12.712352 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:28:12.712352 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:12.712352 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:12.712352 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:12.712352 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:12.712352 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:12.712352 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:12.712352 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:12.712352 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:12.712352 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:12.712352 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:12.712352 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:12.712352 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:12.712352 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:12.712352 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:12.712352 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:12.712352 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:12.713501 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:28:12.713501 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:28:12.713501 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:28:12.713501 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:28:12.713501 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:28:12.713501 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:28:12.713501 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:28:12.713501 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:28:12.713501 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 637/1271] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:28:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:28:12.716570 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566c931a-7b6f-47e6-a5b1-9e2e4ae4b03a None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:12.775036 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:12.818530 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566c931a-7b6f-47e6-a5b1-9e2e4ae4b03a None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:12.855675 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839df9d1-477b-4128-ba57-cf204acc3f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:12.856266 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839df9d1-477b-4128-ba57-cf204acc3f29 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:12.856553 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839df9d1-477b-4128-ba57-cf204acc3f29 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:12.856828 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839df9d1-477b-4128-ba57-cf204acc3f29 None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:12.858258 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839df9d1-477b-4128-ba57-cf204acc3f29 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:12.858747 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839df9d1-477b-4128-ba57-cf204acc3f29 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:12.860222 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839df9d1-477b-4128-ba57-cf204acc3f29 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:12.872754 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 635/1272] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 18 16:28:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:12.878987 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c53c48-94a0-4689-8482-b2d3dae9c861 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:12.971569 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:12.980018 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c53c48-94a0-4689-8482-b2d3dae9c861 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:13.012833 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79236616-5e51-459c-9668-30f6eb0167c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:13.014232 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79236616-5e51-459c-9668-30f6eb0167c4 None admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:13.014510 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79236616-5e51-459c-9668-30f6eb0167c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:13.014823 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79236616-5e51-459c-9668-30f6eb0167c4 None admin] PATH_INFO: `/v3/projects/eb9747e2299c4e0991fd7edb4462c7e5/users/70f704a231a5474e8067a6c2dd88b355/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:13.016882 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79236616-5e51-459c-9668-30f6eb0167c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb9747e2299c4e0991fd7edb4462c7e5, user_id=70f704a231a5474e8067a6c2dd88b355, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:13.018312 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79236616-5e51-459c-9668-30f6eb0167c4 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:13.132354 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 638/1273] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 18 16:28:12 2022] PUT /identity/v3/projects/eb9747e2299c4e0991fd7edb4462c7e5/users/70f704a231a5474e8067a6c2dd88b355/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:28:15.189135 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e6a50-7d32-472d-8caf-0d2b01d4aaea None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:15.189745 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e6a50-7d32-472d-8caf-0d2b01d4aaea None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:15.189844 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e6a50-7d32-472d-8caf-0d2b01d4aaea None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:15.191196 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 636/1274] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:28:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:28:15.196571 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642bed79-facd-424e-99b5-b2c778667d75 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:15.197007 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642bed79-facd-424e-99b5-b2c778667d75 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:15.197142 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642bed79-facd-424e-99b5-b2c778667d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:15.214848 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-642bed79-facd-424e-99b5-b2c778667d75 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:28:15.219887 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-642bed79-facd-424e-99b5-b2c778667d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:28:15.285159 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 639/1275] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:28:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:28:15.310237 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86e6a50-7d32-472d-8caf-0d2b01d4aaea None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:15.418650 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86e6a50-7d32-472d-8caf-0d2b01d4aaea None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:15.440168 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:15.458587 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf600bc4-a665-4c34-af26-3c0e7ac081b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:15.459589 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf600bc4-a665-4c34-af26-3c0e7ac081b4 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:15.466653 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf600bc4-a665-4c34-af26-3c0e7ac081b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:15.466653 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf600bc4-a665-4c34-af26-3c0e7ac081b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:15.471362 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf600bc4-a665-4c34-af26-3c0e7ac081b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:28:15.479176 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-cf600bc4-a665-4c34-af26-3c0e7ac081b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:28:15.479176 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:15.479176 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:15.479176 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:15.479176 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:15.479176 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:15.479176 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:15.479176 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:15.479176 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:15.479176 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:15.479176 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:15.479176 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:15.479176 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:28:15.479176 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:28:15.479176 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:28:15.479176 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:28:15.480473 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:28:15.480473 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:28:15.480473 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:28:15.480473 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:28:15.480473 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:15.480473 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:15.480473 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:15.480473 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:15.480473 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:15.480473 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:15.480473 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:15.480473 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:15.480473 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:15.480473 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:15.480473 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:15.480473 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:15.481737 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:15.481737 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:15.481737 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:15.481737 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:15.481737 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:28:15.481737 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:28:15.481737 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:28:15.481737 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:28:15.481737 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:28:15.481737 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:28:15.481737 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:28:15.481737 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:28:15.481737 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 637/1276] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 18 16:28:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:28:15.487333 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642bed79-facd-424e-99b5-b2c778667d75 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:15.569802 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:15.586931 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642bed79-facd-424e-99b5-b2c778667d75 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:15.620125 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42962d6b-8693-4b62-91f1-26cef52ed730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:15.620786 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42962d6b-8693-4b62-91f1-26cef52ed730 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:15.621067 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42962d6b-8693-4b62-91f1-26cef52ed730 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:15.621341 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42962d6b-8693-4b62-91f1-26cef52ed730 None admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:15.622077 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42962d6b-8693-4b62-91f1-26cef52ed730 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:15.622379 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42962d6b-8693-4b62-91f1-26cef52ed730 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:15.624499 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42962d6b-8693-4b62-91f1-26cef52ed730 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:15.632515 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 640/1277] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 18 16:28:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:15.638321 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf600bc4-a665-4c34-af26-3c0e7ac081b4 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:15.741154 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf600bc4-a665-4c34-af26-3c0e7ac081b4 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:15.774799 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a90ff4-cb0e-41e6-ae93-d1b59f544ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:15.775740 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a90ff4-cb0e-41e6-ae93-d1b59f544ef9 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:15.776025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a90ff4-cb0e-41e6-ae93-d1b59f544ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:15.776303 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a90ff4-cb0e-41e6-ae93-d1b59f544ef9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:15.787549 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a90ff4-cb0e-41e6-ae93-d1b59f544ef9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:15.789013 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a90ff4-cb0e-41e6-ae93-d1b59f544ef9 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:15.806879 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-35a90ff4-cb0e-41e6-ae93-d1b59f544ef9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:28:15.806879 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:15.806879 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:15.806879 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:15.806879 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:15.806879 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:15.806879 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:15.806879 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:15.806879 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:15.806879 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:15.806879 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:15.806879 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:15.806879 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:28:15.806879 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:28:15.806879 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:28:15.808044 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:28:15.808044 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:15.808044 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:15.808044 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:15.808044 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:15.808044 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:15.808044 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:15.808044 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:15.808044 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:15.808044 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:15.808044 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:15.808044 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:15.808044 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:15.808044 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:15.808044 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:15.808044 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:15.808044 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:15.809174 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:28:15.809174 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:28:15.809174 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:28:15.809174 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:28:15.809174 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 638/1278] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:28:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:28:15.814183 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42962d6b-8693-4b62-91f1-26cef52ed730 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:15.925392 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42962d6b-8693-4b62-91f1-26cef52ed730 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:15.962176 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611c0d1e-8a93-43c9-a51d-4a5af53dee42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:15.962856 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c0d1e-8a93-43c9-a51d-4a5af53dee42 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:15.963179 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c0d1e-8a93-43c9-a51d-4a5af53dee42 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:15.963555 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c0d1e-8a93-43c9-a51d-4a5af53dee42 None admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:15.964290 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611c0d1e-8a93-43c9-a51d-4a5af53dee42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:15.964597 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611c0d1e-8a93-43c9-a51d-4a5af53dee42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:15.966003 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611c0d1e-8a93-43c9-a51d-4a5af53dee42 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:15.980201 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 641/1279] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:28:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:15.987128 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a90ff4-cb0e-41e6-ae93-d1b59f544ef9 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:16.020016 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:16.096513 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a90ff4-cb0e-41e6-ae93-d1b59f544ef9 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:16.133862 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698cdccc-0894-4b1d-8d2f-cb16adcf8cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:16.134867 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698cdccc-0894-4b1d-8d2f-cb16adcf8cb1 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:16.135305 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698cdccc-0894-4b1d-8d2f-cb16adcf8cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:16.135616 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698cdccc-0894-4b1d-8d2f-cb16adcf8cb1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:16.144246 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698cdccc-0894-4b1d-8d2f-cb16adcf8cb1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:16.145711 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698cdccc-0894-4b1d-8d2f-cb16adcf8cb1 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:16.162533 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-698cdccc-0894-4b1d-8d2f-cb16adcf8cb1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 16:28:16.162533 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:16.162533 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:16.162533 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:16.162533 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:16.162533 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:16.162533 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:16.162533 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:16.162533 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:16.162533 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:16.162533 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:16.162533 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:16.162533 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:28:16.162533 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:28:16.162533 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:28:16.162533 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:28:16.169238 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:16.169238 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:16.169238 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:28:16.169238 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:28:16.169238 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:28:16.169238 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:28:16.169238 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:16.169238 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:16.169238 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:16.169238 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:16.169238 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:16.169238 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:16.169238 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:16.169238 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:16.169238 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:16.169238 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:16.169238 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:16.169238 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:16.171028 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:16.171028 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:16.171028 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:28:16.171028 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:28:16.171028 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:28:16.171028 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:28:16.171028 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:28:16.171028 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:28:16.171028 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 16:28:16.171028 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:28:16.171028 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 639/1280] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:28:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:28:16.171699 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611c0d1e-8a93-43c9-a51d-4a5af53dee42 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:16.174901 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:16.281798 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611c0d1e-8a93-43c9-a51d-4a5af53dee42 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:16.317771 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1446eea-c413-4373-997b-68b61cdec26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:16.318486 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1446eea-c413-4373-997b-68b61cdec26e None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:16.318772 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1446eea-c413-4373-997b-68b61cdec26e None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:16.319050 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1446eea-c413-4373-997b-68b61cdec26e None admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:16.319923 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1446eea-c413-4373-997b-68b61cdec26e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:16.320230 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1446eea-c413-4373-997b-68b61cdec26e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:16.321676 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1446eea-c413-4373-997b-68b61cdec26e None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:16.340128 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 642/1281] 10.222.0.26 () {56 vars in 1143 bytes} [Mon Jul 18 16:28:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:16.346553 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698cdccc-0894-4b1d-8d2f-cb16adcf8cb1 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:16.437835 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:16.447118 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698cdccc-0894-4b1d-8d2f-cb16adcf8cb1 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:16.480671 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e113cf7f-54cf-493c-8c1d-1f67410aca4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:16.481319 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113cf7f-54cf-493c-8c1d-1f67410aca4e None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:16.481602 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113cf7f-54cf-493c-8c1d-1f67410aca4e None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:16.481876 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113cf7f-54cf-493c-8c1d-1f67410aca4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:16.548081 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e113cf7f-54cf-493c-8c1d-1f67410aca4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:16.549455 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e113cf7f-54cf-493c-8c1d-1f67410aca4e None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:16.652067 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:16.662284 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 640/1282] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 18 16:28:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 16:28:16.668746 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1446eea-c413-4373-997b-68b61cdec26e None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:16.734861 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:16.773990 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1446eea-c413-4373-997b-68b61cdec26e None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:16.808065 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a2f722-f1b6-42bc-ae94-0451343ca131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:16.808770 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a2f722-f1b6-42bc-ae94-0451343ca131 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:16.809052 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a2f722-f1b6-42bc-ae94-0451343ca131 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:16.809325 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a2f722-f1b6-42bc-ae94-0451343ca131 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:16.821617 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a2f722-f1b6-42bc-ae94-0451343ca131 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:16.823049 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a2f722-f1b6-42bc-ae94-0451343ca131 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:16.840892 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-89a2f722-f1b6-42bc-ae94-0451343ca131 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:28:16.840892 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:16.840892 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:16.840892 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:16.840892 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:16.840892 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:16.840892 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:16.840892 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:16.840892 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:16.840892 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:16.840892 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:16.840892 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:16.840892 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:28:16.840892 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:28:16.840892 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:28:16.842387 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:28:16.842387 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:16.842387 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:16.842387 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:16.842387 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:16.842387 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:16.842387 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:16.842387 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:16.842387 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:16.842387 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:16.842387 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:16.842387 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:16.842387 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:16.842387 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:16.842387 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:16.842387 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:16.842387 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:16.843286 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:28:16.843286 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:28:16.843286 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:28:16.843286 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:28:16.843286 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 643/1283] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:28:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:28:16.848381 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e113cf7f-54cf-493c-8c1d-1f67410aca4e None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:16.954773 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e113cf7f-54cf-493c-8c1d-1f67410aca4e None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:16.988880 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3551f25-7bfd-4f6c-ae79-b33c5aa49a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:16.989529 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3551f25-7bfd-4f6c-ae79-b33c5aa49a99 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:16.989810 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3551f25-7bfd-4f6c-ae79-b33c5aa49a99 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:16.990082 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3551f25-7bfd-4f6c-ae79-b33c5aa49a99 None admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:16.990806 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3551f25-7bfd-4f6c-ae79-b33c5aa49a99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:16.991106 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3551f25-7bfd-4f6c-ae79-b33c5aa49a99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:16.992505 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3551f25-7bfd-4f6c-ae79-b33c5aa49a99 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:17.004077 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 641/1284] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 18 16:28:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:17.009949 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a2f722-f1b6-42bc-ae94-0451343ca131 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:17.111604 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a2f722-f1b6-42bc-ae94-0451343ca131 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:17.126089 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:17.145764 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab389d80-502d-4f8b-ad50-470b5010d055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:17.146560 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab389d80-502d-4f8b-ad50-470b5010d055 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:17.146846 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab389d80-502d-4f8b-ad50-470b5010d055 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:17.147124 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab389d80-502d-4f8b-ad50-470b5010d055 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:17.153675 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab389d80-502d-4f8b-ad50-470b5010d055 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:17.155948 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab389d80-502d-4f8b-ad50-470b5010d055 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:17.171445 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-ab389d80-502d-4f8b-ad50-470b5010d055 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:28:17.171445 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:17.171445 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:17.171445 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:17.171445 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:17.171445 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:17.171445 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:17.171445 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:17.171445 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:17.171445 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:17.171445 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:17.171445 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:17.171445 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:28:17.171445 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:28:17.171445 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:28:17.172534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:28:17.172534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:17.172534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:17.172534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:17.172534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:17.172534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:17.172534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:17.172534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:17.172534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:17.172534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:17.172534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:17.172534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:17.172534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:17.172534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:17.172534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:17.172534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:17.172534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:17.173492 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:28:17.173492 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:28:17.173492 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:28:17.173492 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:28:17.173492 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:28:17.173492 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:28:17.173492 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:28:17.173492 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 16:28:17.173492 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 644/1285] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 18 16:28:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:28:17.178765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3551f25-7bfd-4f6c-ae79-b33c5aa49a99 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:17.229710 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:17.280774 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3551f25-7bfd-4f6c-ae79-b33c5aa49a99 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:17.318273 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d764519b-dcd9-46d8-bc9a-ac3fd8bae128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:17.318976 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d764519b-dcd9-46d8-bc9a-ac3fd8bae128 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:17.319247 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d764519b-dcd9-46d8-bc9a-ac3fd8bae128 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:17.319550 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d764519b-dcd9-46d8-bc9a-ac3fd8bae128 None admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:17.320342 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d764519b-dcd9-46d8-bc9a-ac3fd8bae128 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:17.320654 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d764519b-dcd9-46d8-bc9a-ac3fd8bae128 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:17.322091 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d764519b-dcd9-46d8-bc9a-ac3fd8bae128 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:17.335589 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 642/1286] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 18 16:28:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:17.342132 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab389d80-502d-4f8b-ad50-470b5010d055 None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:17.443464 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab389d80-502d-4f8b-ad50-470b5010d055 None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:17.477136 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d4bbf7-70e3-4763-9574-6ca7e9ea4999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:17.477785 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d4bbf7-70e3-4763-9574-6ca7e9ea4999 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:17.478064 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d4bbf7-70e3-4763-9574-6ca7e9ea4999 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:17.478337 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d4bbf7-70e3-4763-9574-6ca7e9ea4999 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:17.479160 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d4bbf7-70e3-4763-9574-6ca7e9ea4999 None admin] RBAC: Adding query filter params (role.id=98eaadb3669f4b608d572c001742f559, scope.project.id=eb9747e2299c4e0991fd7edb4462c7e5, user.id=70f704a231a5474e8067a6c2dd88b355) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:17.479466 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d4bbf7-70e3-4763-9574-6ca7e9ea4999 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:17.480928 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d4bbf7-70e3-4763-9574-6ca7e9ea4999 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:17.487852 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 645/1287] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 18 16:28:17 2022] GET /identity/v3/role_assignments?user.id=70f704a231a5474e8067a6c2dd88b355&scope.project.id=eb9747e2299c4e0991fd7edb4462c7e5&role.id=98eaadb3669f4b608d572c001742f559 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:19.552455 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e835f0-fc2e-488c-8e31-d40e7fe9fd5b None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:19.553021 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e835f0-fc2e-488c-8e31-d40e7fe9fd5b None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:19.553021 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e835f0-fc2e-488c-8e31-d40e7fe9fd5b None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:19.554372 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 643/1288] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:28:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:28:19.559598 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6715416c-807d-4c45-8937-b48f31c0f3ec None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:19.560041 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6715416c-807d-4c45-8937-b48f31c0f3ec None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:19.560143 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6715416c-807d-4c45-8937-b48f31c0f3ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:19.577935 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-6715416c-807d-4c45-8937-b48f31c0f3ec None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:28:19.579149 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-6715416c-807d-4c45-8937-b48f31c0f3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:28:19.610549 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:19.648997 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 646/1289] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:28:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:28:19.673274 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e835f0-fc2e-488c-8e31-d40e7fe9fd5b None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:19.775559 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e835f0-fc2e-488c-8e31-d40e7fe9fd5b None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:19.786776 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:19.811675 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110a9792-18ff-4e1a-98ab-ba48187c4b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:19.812346 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a9792-18ff-4e1a-98ab-ba48187c4b0a None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:19.812680 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a9792-18ff-4e1a-98ab-ba48187c4b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:19.812952 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a9792-18ff-4e1a-98ab-ba48187c4b0a None admin] PATH_INFO: `/v3/services/placement` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:19.813609 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110a9792-18ff-4e1a-98ab-ba48187c4b0a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:19.815101 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110a9792-18ff-4e1a-98ab-ba48187c4b0a None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:19.827329 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-110a9792-18ff-4e1a-98ab-ba48187c4b0a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 16:28:19.827329 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:19.827329 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:28:19.827329 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:28:19.827329 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:28:19.827329 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:28:19.827329 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:28:19.827329 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:28:19.827329 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 16:28:19.827329 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:28:19.827329 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:28:19.827329 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:28:19.827329 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:19.827329 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:19.827329 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:19.827329 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:19.827329 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:19.828507 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:19.828507 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:19.828507 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:19.828507 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:19.828507 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:19.828507 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:19.828507 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:28:19.828507 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:28:19.828507 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:28:19.828507 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:28:19.828507 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:19.828507 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:19.828507 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:19.828507 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:19.828507 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:19.828507 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:19.833753 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:19.833753 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:19.833753 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:19.833753 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:19.833753 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:19.833753 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:19.833753 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:19.833753 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:19.833753 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:19.833753 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:19.833753 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:28:19.833753 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:28:19.833753 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 16:28:19.833753 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:28:19.833753 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 644/1290] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Jul 18 16:28:19 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:28:19.834854 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6715416c-807d-4c45-8937-b48f31c0f3ec None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:19.933312 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6715416c-807d-4c45-8937-b48f31c0f3ec None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:19.968109 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edf9b0e-c3b7-486b-be17-a1de6b0589e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:19.968775 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf9b0e-c3b7-486b-be17-a1de6b0589e2 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:19.969059 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf9b0e-c3b7-486b-be17-a1de6b0589e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:19.969329 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf9b0e-c3b7-486b-be17-a1de6b0589e2 None admin] PATH_INFO: `/v3/services` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:19.970027 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edf9b0e-c3b7-486b-be17-a1de6b0589e2 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:19.970321 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edf9b0e-c3b7-486b-be17-a1de6b0589e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:19.971722 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edf9b0e-c3b7-486b-be17-a1de6b0589e2 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:19.979369 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 647/1291] 10.222.0.26 () {56 vars in 1116 bytes} [Mon Jul 18 16:28:19 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:19.985578 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110a9792-18ff-4e1a-98ab-ba48187c4b0a None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:20.087161 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110a9792-18ff-4e1a-98ab-ba48187c4b0a None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:20.120740 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec6472-04e1-4074-9fd4-b3964c0f8d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:20.121407 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec6472-04e1-4074-9fd4-b3964c0f8d5b None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:20.121693 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec6472-04e1-4074-9fd4-b3964c0f8d5b None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:20.121970 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec6472-04e1-4074-9fd4-b3964c0f8d5b None admin] PATH_INFO: `/v3/services` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:20.122674 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cec6472-04e1-4074-9fd4-b3964c0f8d5b None admin] RBAC: Adding query filter params (type=placement) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:20.124092 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cec6472-04e1-4074-9fd4-b3964c0f8d5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:20.125300 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cec6472-04e1-4074-9fd4-b3964c0f8d5b None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:20.130049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 645/1292] 10.222.0.26 () {56 vars in 1116 bytes} [Mon Jul 18 16:28:19 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:28:22.176018 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7681f3ee-a94b-430a-bde7-d4128cab4b81 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:22.176513 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7681f3ee-a94b-430a-bde7-d4128cab4b81 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:22.176513 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7681f3ee-a94b-430a-bde7-d4128cab4b81 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:22.177874 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 648/1293] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:28:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:28:22.183153 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d615c-b607-457c-8727-1ceb2c3a6100 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:22.183570 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d615c-b607-457c-8727-1ceb2c3a6100 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:22.183749 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d615c-b607-457c-8727-1ceb2c3a6100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:22.200542 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-8e7d615c-b607-457c-8727-1ceb2c3a6100 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:28:22.202488 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-8e7d615c-b607-457c-8727-1ceb2c3a6100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:28:22.234413 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:22.272514 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 646/1294] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:28:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:28:22.296483 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7681f3ee-a94b-430a-bde7-d4128cab4b81 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:22.328513 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:22.400761 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7681f3ee-a94b-430a-bde7-d4128cab4b81 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:22.434942 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf3fa01-b17d-4d63-8b54-82bd5bebf4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:22.436668 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf3fa01-b17d-4d63-8b54-82bd5bebf4b4 None admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:22.436961 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf3fa01-b17d-4d63-8b54-82bd5bebf4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:22.437235 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf3fa01-b17d-4d63-8b54-82bd5bebf4b4 None admin] PATH_INFO: `/v3/services` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:22.437949 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf3fa01-b17d-4d63-8b54-82bd5bebf4b4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:22.439310 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf3fa01-b17d-4d63-8b54-82bd5bebf4b4 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:22.566515 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 649/1295] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jul 18 16:28:22 2022] POST /identity/v3/services => generated 246 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:28:24.709243 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd7512-56dd-4ad7-95e0-f39f96749a0c None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:24.709894 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd7512-56dd-4ad7-95e0-f39f96749a0c None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:24.709894 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd7512-56dd-4ad7-95e0-f39f96749a0c None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:24.711321 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 647/1296] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:28:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:28:24.717570 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618765db-6059-4eb5-9e6f-760e9eb015a5 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:24.718281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618765db-6059-4eb5-9e6f-760e9eb015a5 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:24.718402 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618765db-6059-4eb5-9e6f-760e9eb015a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:24.737649 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-618765db-6059-4eb5-9e6f-760e9eb015a5 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:28:24.739029 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-618765db-6059-4eb5-9e6f-760e9eb015a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:28:24.856099 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 650/1297] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:28:24 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:28:24.880030 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cd7512-56dd-4ad7-95e0-f39f96749a0c None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:24.996019 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cd7512-56dd-4ad7-95e0-f39f96749a0c None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:25.033784 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a64341-78aa-4869-98f6-40caf9f79442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:25.034600 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a64341-78aa-4869-98f6-40caf9f79442 None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:25.034884 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a64341-78aa-4869-98f6-40caf9f79442 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:25.035160 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a64341-78aa-4869-98f6-40caf9f79442 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:25.035848 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a64341-78aa-4869-98f6-40caf9f79442 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:25.037914 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a64341-78aa-4869-98f6-40caf9f79442 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:25.051011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-37a64341-78aa-4869-98f6-40caf9f79442 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 16:28:25.051011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:25.051011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:28:25.051011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:28:25.051011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:28:25.051011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:28:25.051011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:28:25.051011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:28:25.051011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 16:28:25.051011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:28:25.051011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:28:25.051011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:28:25.051011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:25.051011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:25.051011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:25.051011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:25.051011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:25.053047 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:25.053047 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:25.053047 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:25.053047 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:25.053047 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:25.053047 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:25.053047 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:28:25.053047 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:28:25.053047 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:28:25.053047 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:28:25.053047 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:25.053047 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:25.053047 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:25.053047 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:25.053047 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:25.053047 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:25.054756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:25.054756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:25.054756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:25.054756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:25.054756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:25.054756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:25.054756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:25.054756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:25.054756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:25.054756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:25.054756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:28:25.054756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:28:25.054756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 16:28:25.054756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:28:25.054756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 648/1298] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Jul 18 16:28:24 2022] GET /identity/v3/services/placement => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:28:25.059551 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618765db-6059-4eb5-9e6f-760e9eb015a5 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:25.085763 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:25.176736 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618765db-6059-4eb5-9e6f-760e9eb015a5 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:25.213633 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac365a9-59e6-4676-9104-55bb6ceee8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:25.214367 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac365a9-59e6-4676-9104-55bb6ceee8df None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:25.214708 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac365a9-59e6-4676-9104-55bb6ceee8df None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:25.215014 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac365a9-59e6-4676-9104-55bb6ceee8df None admin] PATH_INFO: `/v3/services` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:25.215910 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac365a9-59e6-4676-9104-55bb6ceee8df None admin] RBAC: Adding query filter params (name=placement) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:25.216254 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac365a9-59e6-4676-9104-55bb6ceee8df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:25.217718 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac365a9-59e6-4676-9104-55bb6ceee8df None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:25.224557 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 651/1299] 10.222.0.26 () {56 vars in 1116 bytes} [Mon Jul 18 16:28:25 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:25.231211 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a64341-78aa-4869-98f6-40caf9f79442 None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:25.276407 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:25.352929 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a64341-78aa-4869-98f6-40caf9f79442 None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:25.401863 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a157c608-9ec3-47ff-824f-2d5f4eecf51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:25.402600 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a157c608-9ec3-47ff-824f-2d5f4eecf51c None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:25.402928 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a157c608-9ec3-47ff-824f-2d5f4eecf51c None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:25.403245 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a157c608-9ec3-47ff-824f-2d5f4eecf51c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:25.404739 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a157c608-9ec3-47ff-824f-2d5f4eecf51c None admin] RBAC: Adding query filter params (interface=public, service_id=7946ad0438134e42a29120ca11cc92ba, region_id=RegionOne) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:25.405087 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a157c608-9ec3-47ff-824f-2d5f4eecf51c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:25.406771 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a157c608-9ec3-47ff-824f-2d5f4eecf51c None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:25.412629 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 649/1300] 10.222.0.26 () {56 vars in 1251 bytes} [Mon Jul 18 16:28:25 2022] GET /identity/v3/endpoints?service_id=7946ad0438134e42a29120ca11cc92ba&interface=public®ion_id=RegionOne => generated 194 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:28:25.419260 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac365a9-59e6-4676-9104-55bb6ceee8df None admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:25.515138 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:25.549690 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac365a9-59e6-4676-9104-55bb6ceee8df None admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:25.613882 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d250b2-6e16-4df1-aad8-bb7081ccfc43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:25.615023 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d250b2-6e16-4df1-aad8-bb7081ccfc43 None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:25.615553 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d250b2-6e16-4df1-aad8-bb7081ccfc43 None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:25.616037 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d250b2-6e16-4df1-aad8-bb7081ccfc43 None admin] PATH_INFO: `/v3/services` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:25.617307 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d250b2-6e16-4df1-aad8-bb7081ccfc43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:25.619799 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d250b2-6e16-4df1-aad8-bb7081ccfc43 None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:25.634768 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 652/1301] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Jul 18 16:28:25 2022] GET /identity/v3/services => generated 2191 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:28:28.800093 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b05e19-7d16-4d87-b5a7-59052efae03c None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:28.800984 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b05e19-7d16-4d87-b5a7-59052efae03c None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:28.800984 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b05e19-7d16-4d87-b5a7-59052efae03c None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:28.802514 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 650/1302] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:28:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:28:28.808451 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79a586-37f0-43e3-8647-ffcc37b71444 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:28.808946 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79a586-37f0-43e3-8647-ffcc37b71444 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:28.815800 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79a586-37f0-43e3-8647-ffcc37b71444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:28.830421 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-0d79a586-37f0-43e3-8647-ffcc37b71444 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:28:28.831707 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-0d79a586-37f0-43e3-8647-ffcc37b71444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:28:28.910879 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 653/1303] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:28:28 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:28:28.934378 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b05e19-7d16-4d87-b5a7-59052efae03c None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:29.055153 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:29.068342 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b05e19-7d16-4d87-b5a7-59052efae03c None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:29.120447 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6bc966-a719-4588-9b33-00510896a8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:29.121274 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6bc966-a719-4588-9b33-00510896a8fd None admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:29.121639 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6bc966-a719-4588-9b33-00510896a8fd None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:29.121917 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6bc966-a719-4588-9b33-00510896a8fd None admin] PATH_INFO: `/v3/services/placement` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:29.122685 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6bc966-a719-4588-9b33-00510896a8fd None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:29.126100 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6bc966-a719-4588-9b33-00510896a8fd None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:29.140486 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-1c6bc966-a719-4588-9b33-00510896a8fd None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 16:28:29.140486 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:29.140486 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:28:29.140486 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:28:29.140486 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:28:29.140486 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:28:29.140486 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:28:29.140486 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:28:29.140486 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 16:28:29.140486 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:28:29.140486 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:28:29.140486 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:28:29.140486 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:28:29.140486 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:28:29.140486 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:28:29.140486 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:28:29.140486 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:28:29.142245 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:28:29.142245 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:28:29.142245 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:28:29.142245 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:28:29.142245 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:28:29.142245 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:28:29.142245 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:28:29.142245 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:28:29.142245 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:28:29.142245 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:28:29.142245 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:28:29.142245 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:28:29.142245 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:28:29.142245 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:28:29.142245 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:28:29.142245 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:28:29.143691 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:28:29.143691 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 16:28:29.143691 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:28:29.143691 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:28:29.143691 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:28:29.143691 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:28:29.143691 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:28:29.143691 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:28:29.143691 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:28:29.143691 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:28:29.143691 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:28:29.143691 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:28:29.143691 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 16:28:29.143691 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 16:28:29.143691 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 651/1304] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Jul 18 16:28:28 2022] GET /identity/v3/services/placement => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:28:29.148260 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d79a586-37f0-43e3-8647-ffcc37b71444 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:29.235707 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:29.261317 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d79a586-37f0-43e3-8647-ffcc37b71444 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:29.303185 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76290c9-12d1-477b-8761-2059d4a7cd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:29.303920 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76290c9-12d1-477b-8761-2059d4a7cd9b None admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:29.304206 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76290c9-12d1-477b-8761-2059d4a7cd9b None admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:29.304478 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76290c9-12d1-477b-8761-2059d4a7cd9b None admin] PATH_INFO: `/v3/services` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:29.305271 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76290c9-12d1-477b-8761-2059d4a7cd9b None admin] RBAC: Adding query filter params (name=placement) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:28:29.305520 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76290c9-12d1-477b-8761-2059d4a7cd9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:29.306922 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76290c9-12d1-477b-8761-2059d4a7cd9b None admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:29.312536 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 654/1305] 10.222.0.26 () {56 vars in 1116 bytes} [Mon Jul 18 16:28:29 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:28:29.318794 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6bc966-a719-4588-9b33-00510896a8fd None admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:28:29.427775 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6bc966-a719-4588-9b33-00510896a8fd None admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:28:29.463456 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aa439d-9648-45d5-9089-f093525dd032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:28:29.464148 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aa439d-9648-45d5-9089-f093525dd032 None admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:29.464462 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aa439d-9648-45d5-9089-f093525dd032 None admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:29.464723 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aa439d-9648-45d5-9089-f093525dd032 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:29.465471 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aa439d-9648-45d5-9089-f093525dd032 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:28:29.467776 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aa439d-9648-45d5-9089-f093525dd032 None admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:28:29.648899 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 652/1306] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Jul 18 16:28:29 2022] POST /identity/v3/endpoints => generated 331 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:28:39.309240 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=30 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:28:39.641360 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:36:51.248172 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e62a83-a9e9-4db5-9190-e79203db11bc None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:36:51.249199 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e62a83-a9e9-4db5-9190-e79203db11bc None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:36:51.249694 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e62a83-a9e9-4db5-9190-e79203db11bc None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:36:51.252042 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 655/1307] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:36:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:36:51.257254 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b238d4-6406-4498-bb84-91b64e2cb176 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:36:51.257743 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b238d4-6406-4498-bb84-91b64e2cb176 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:36:51.257837 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b238d4-6406-4498-bb84-91b64e2cb176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:36:51.353647 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-f6b238d4-6406-4498-bb84-91b64e2cb176 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:36:51.354915 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f6b238d4-6406-4498-bb84-91b64e2cb176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:36:51.503399 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 653/1308] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:36:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:36:51.539356 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f8f7b-640f-4af4-b619-2a4653c5649d None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:36:51.539688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f8f7b-640f-4af4-b619-2a4653c5649d None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:36:51.539982 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f8f7b-640f-4af4-b619-2a4653c5649d None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:36:51.541552 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 656/1309] 10.222.0.26 () {54 vars in 965 bytes} [Mon Jul 18 16:36:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:36:51.547624 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76b8ce8-33bc-4999-8e28-0bfcd753948a None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:36:51.547826 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76b8ce8-33bc-4999-8e28-0bfcd753948a None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:36:51.548106 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76b8ce8-33bc-4999-8e28-0bfcd753948a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:36:51.646229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-a76b8ce8-33bc-4999-8e28-0bfcd753948a None None] MFA Rules not processed for user `7a6054dbfbbc487794d2a32061b0ac82`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:36:51.647386 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a76b8ce8-33bc-4999-8e28-0bfcd753948a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:36:51.669854 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:36:51.731395 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 654/1310] 10.222.0.26 () {58 vars in 1062 bytes} [Mon Jul 18 16:36:51 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:36:51.740869 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81f8f7b-640f-4af4-b619-2a4653c5649d None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:36:51.742691 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-d81f8f7b-640f-4af4-b619-2a4653c5649d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:36:51.867664 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81f8f7b-640f-4af4-b619-2a4653c5649d None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:36:51.951218 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13691d85-77f8-4bc3-be3d-528c7a608791 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:36:51.951872 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13691d85-77f8-4bc3-be3d-528c7a608791 service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:36:51.952159 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13691d85-77f8-4bc3-be3d-528c7a608791 service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:36:51.952433 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13691d85-77f8-4bc3-be3d-528c7a608791 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:36:51.954100 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-13691d85-77f8-4bc3-be3d-528c7a608791 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:36:52.014052 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:36:52.071381 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13691d85-77f8-4bc3-be3d-528c7a608791 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:36:52.073573 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13691d85-77f8-4bc3-be3d-528c7a608791 service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:36:52.092867 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 657/1311] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:36:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:36:54.469870 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c1b34-aee2-476a-888e-4eedeff0d284 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:36:54.470439 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c1b34-aee2-476a-888e-4eedeff0d284 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:36:54.470439 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c1b34-aee2-476a-888e-4eedeff0d284 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:36:54.472162 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 655/1312] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:36:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:36:54.478091 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6d473-1d2f-40ac-83ae-2216610cef52 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:36:54.478527 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6d473-1d2f-40ac-83ae-2216610cef52 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:36:54.478786 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6d473-1d2f-40ac-83ae-2216610cef52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:36:54.551186 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-bce6d473-1d2f-40ac-83ae-2216610cef52 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:36:54.552470 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-bce6d473-1d2f-40ac-83ae-2216610cef52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:36:54.656978 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 658/1313] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:36:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:36:54.714421 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b071c217-691d-471c-b88e-d647d8add88f None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:36:54.714774 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b071c217-691d-471c-b88e-d647d8add88f None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:36:54.714980 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b071c217-691d-471c-b88e-d647d8add88f None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:36:54.716829 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 656/1314] 10.222.0.26 () {54 vars in 965 bytes} [Mon Jul 18 16:36:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:36:54.723384 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e762947-2072-44c2-ae47-6e1156afb6b3 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:36:54.723747 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e762947-2072-44c2-ae47-6e1156afb6b3 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:36:54.723889 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e762947-2072-44c2-ae47-6e1156afb6b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:36:54.735820 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:36:54.812199 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-4e762947-2072-44c2-ae47-6e1156afb6b3 None None] MFA Rules not processed for user `7a6054dbfbbc487794d2a32061b0ac82`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:36:54.813448 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-4e762947-2072-44c2-ae47-6e1156afb6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:36:54.900509 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 659/1315] 10.222.0.26 () {58 vars in 1062 bytes} [Mon Jul 18 16:36:54 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:36:54.909107 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b071c217-691d-471c-b88e-d647d8add88f None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:36:54.910758 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-b071c217-691d-471c-b88e-d647d8add88f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:36:55.018167 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b071c217-691d-471c-b88e-d647d8add88f None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:36:55.095505 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:36:55.098243 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb1ab54-09d8-41f5-99e6-bb54c0ace959 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:36:55.098893 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1ab54-09d8-41f5-99e6-bb54c0ace959 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:36:55.099175 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1ab54-09d8-41f5-99e6-bb54c0ace959 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:36:55.099452 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1ab54-09d8-41f5-99e6-bb54c0ace959 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:36:55.101136 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-2fb1ab54-09d8-41f5-99e6-bb54c0ace959 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:36:55.217695 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb1ab54-09d8-41f5-99e6-bb54c0ace959 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:36:55.219143 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb1ab54-09d8-41f5-99e6-bb54c0ace959 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:36:55.238359 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 657/1316] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:36:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:36:58.094737 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdba835-52f9-47f4-abdf-bb90893d48d8 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:36:58.095324 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdba835-52f9-47f4-abdf-bb90893d48d8 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:36:58.095324 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdba835-52f9-47f4-abdf-bb90893d48d8 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:36:58.096709 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 660/1317] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:36:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:36:58.101792 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b6a4f-75e4-4cdb-9f83-16add302e493 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:36:58.106278 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b6a4f-75e4-4cdb-9f83-16add302e493 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:36:58.106278 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b6a4f-75e4-4cdb-9f83-16add302e493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:36:58.168684 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-1f2b6a4f-75e4-4cdb-9f83-16add302e493 None None] MFA Rules not processed for user `cda57e656e834b9792465e79b138c9c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:36:58.169799 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-1f2b6a4f-75e4-4cdb-9f83-16add302e493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:36:58.222734 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:36:58.266854 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 658/1318] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:36:58 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:36:58.308802 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdba835-52f9-47f4-abdf-bb90893d48d8 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:36:58.310334 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-bcdba835-52f9-47f4-abdf-bb90893d48d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:36:58.413563 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdba835-52f9-47f4-abdf-bb90893d48d8 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:36:58.417322 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:36:58.489367 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427d078-2203-49d0-9dd9-ad0ef5b5fd98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:36:58.489936 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427d078-2203-49d0-9dd9-ad0ef5b5fd98 service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:36:58.490222 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427d078-2203-49d0-9dd9-ad0ef5b5fd98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:36:58.490500 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427d078-2203-49d0-9dd9-ad0ef5b5fd98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:36:58.492186 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-f427d078-2203-49d0-9dd9-ad0ef5b5fd98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:36:58.611441 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f427d078-2203-49d0-9dd9-ad0ef5b5fd98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:36:58.612982 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f427d078-2203-49d0-9dd9-ad0ef5b5fd98 service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:36:58.631751 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 661/1319] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:36:58 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:37:01.732772 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51285791-d0e2-4c0d-b7b0-4b9f20a213f3 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:01.733338 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51285791-d0e2-4c0d-b7b0-4b9f20a213f3 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:01.733338 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51285791-d0e2-4c0d-b7b0-4b9f20a213f3 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:01.734722 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 659/1320] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:37:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:37:01.739867 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d3780-614a-4c73-860c-3d4d6673c840 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:01.740306 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d3780-614a-4c73-860c-3d4d6673c840 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:01.740422 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d3780-614a-4c73-860c-3d4d6673c840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:01.779220 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:37:01.804418 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-aa4d3780-614a-4c73-860c-3d4d6673c840 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:37:01.805915 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-aa4d3780-614a-4c73-860c-3d4d6673c840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:01.902223 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 662/1321] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:37:01 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:37:01.941125 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51285791-d0e2-4c0d-b7b0-4b9f20a213f3 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:37:01.942666 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-51285791-d0e2-4c0d-b7b0-4b9f20a213f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:02.048992 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51285791-d0e2-4c0d-b7b0-4b9f20a213f3 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:37:02.123881 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4689ae8e-7409-4df9-9fc9-7452341da838 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:37:02.124545 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4689ae8e-7409-4df9-9fc9-7452341da838 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:02.124826 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4689ae8e-7409-4df9-9fc9-7452341da838 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:02.125099 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4689ae8e-7409-4df9-9fc9-7452341da838 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:02.126746 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-4689ae8e-7409-4df9-9fc9-7452341da838 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:02.135469 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:37:02.241467 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4689ae8e-7409-4df9-9fc9-7452341da838 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:37:02.243049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4689ae8e-7409-4df9-9fc9-7452341da838 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:37:02.262013 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 660/1322] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:37:01 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:37:05.765467 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97496dca-04c8-4900-ba3d-07589186d347 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:05.766036 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97496dca-04c8-4900-ba3d-07589186d347 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:05.766221 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97496dca-04c8-4900-ba3d-07589186d347 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:05.767553 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 663/1323] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:37:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:37:05.772902 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e1c9fe-06db-4d76-ad9a-230fb5b3c6f2 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:05.773354 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e1c9fe-06db-4d76-ad9a-230fb5b3c6f2 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:05.773455 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e1c9fe-06db-4d76-ad9a-230fb5b3c6f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:05.840108 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-53e1c9fe-06db-4d76-ad9a-230fb5b3c6f2 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:37:05.841319 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-53e1c9fe-06db-4d76-ad9a-230fb5b3c6f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:05.900004 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:37:05.939617 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 661/1324] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:37:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:37:05.976853 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97496dca-04c8-4900-ba3d-07589186d347 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:37:05.978611 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-97496dca-04c8-4900-ba3d-07589186d347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:06.094349 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97496dca-04c8-4900-ba3d-07589186d347 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:37:06.108050 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:37:06.176049 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b233bd1d-806f-45b2-83bc-b7b63e51b7c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:37:06.176714 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b233bd1d-806f-45b2-83bc-b7b63e51b7c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:06.177038 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b233bd1d-806f-45b2-83bc-b7b63e51b7c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:06.177283 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b233bd1d-806f-45b2-83bc-b7b63e51b7c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:06.179030 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-b233bd1d-806f-45b2-83bc-b7b63e51b7c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:06.297383 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b233bd1d-806f-45b2-83bc-b7b63e51b7c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:37:06.298827 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b233bd1d-806f-45b2-83bc-b7b63e51b7c6 service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:37:06.317903 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 664/1325] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:37:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:37:08.433465 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c08f63-4ba5-4009-ae50-b6d3468490ac None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:08.433996 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c08f63-4ba5-4009-ae50-b6d3468490ac None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:08.434133 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c08f63-4ba5-4009-ae50-b6d3468490ac None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:08.435544 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 662/1326] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:37:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:37:08.440897 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1de08b-85a0-40d2-a37e-c07008e1bda9 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:08.441307 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1de08b-85a0-40d2-a37e-c07008e1bda9 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:08.441457 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1de08b-85a0-40d2-a37e-c07008e1bda9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:08.490100 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:37:08.508042 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-6f1de08b-85a0-40d2-a37e-c07008e1bda9 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:37:08.511755 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-6f1de08b-85a0-40d2-a37e-c07008e1bda9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:08.609989 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 665/1327] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:37:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:37:08.652340 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c08f63-4ba5-4009-ae50-b6d3468490ac None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:37:08.653952 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-16c08f63-4ba5-4009-ae50-b6d3468490ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:08.757751 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c08f63-4ba5-4009-ae50-b6d3468490ac None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:37:08.836642 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2be8e9e-1ac3-4ffb-ad0f-69e14372d6e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:37:08.837315 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be8e9e-1ac3-4ffb-ad0f-69e14372d6e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:08.837615 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be8e9e-1ac3-4ffb-ad0f-69e14372d6e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:08.837886 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be8e9e-1ac3-4ffb-ad0f-69e14372d6e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:08.839560 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-c2be8e9e-1ac3-4ffb-ad0f-69e14372d6e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:08.856896 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:37:08.959776 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2be8e9e-1ac3-4ffb-ad0f-69e14372d6e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:37:08.961885 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2be8e9e-1ac3-4ffb-ad0f-69e14372d6e6 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:37:08.982792 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 663/1328] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:37:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:37:11.090903 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99254e3-5a09-427a-83b3-45fd3194310f None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:11.091414 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99254e3-5a09-427a-83b3-45fd3194310f None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:11.091531 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99254e3-5a09-427a-83b3-45fd3194310f None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:11.093002 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 666/1329] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:37:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:37:11.098268 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0904440-9366-44c8-8dfc-20c17b45a968 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:11.098722 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0904440-9366-44c8-8dfc-20c17b45a968 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:11.098937 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0904440-9366-44c8-8dfc-20c17b45a968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:11.171016 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-c0904440-9366-44c8-8dfc-20c17b45a968 None None] MFA Rules not processed for user `cda57e656e834b9792465e79b138c9c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:37:11.172344 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-c0904440-9366-44c8-8dfc-20c17b45a968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:11.242601 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:37:11.275295 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 664/1330] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:37:11 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:37:11.313564 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99254e3-5a09-427a-83b3-45fd3194310f None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:37:11.315154 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-e99254e3-5a09-427a-83b3-45fd3194310f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:11.429007 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99254e3-5a09-427a-83b3-45fd3194310f None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:37:11.452241 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:37:11.507711 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cc747b-0ed9-40c1-9051-9645ac7c4809 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:37:11.509336 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc747b-0ed9-40c1-9051-9645ac7c4809 service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:11.509624 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc747b-0ed9-40c1-9051-9645ac7c4809 service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:11.509999 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc747b-0ed9-40c1-9051-9645ac7c4809 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:11.511647 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-86cc747b-0ed9-40c1-9051-9645ac7c4809 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:11.626492 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cc747b-0ed9-40c1-9051-9645ac7c4809 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:37:11.628016 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cc747b-0ed9-40c1-9051-9645ac7c4809 service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:37:11.647781 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 667/1331] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:37:11 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:37:14.183805 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d4b90-a634-47ce-a559-4c407f29c205 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:14.184225 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d4b90-a634-47ce-a559-4c407f29c205 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:14.184361 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d4b90-a634-47ce-a559-4c407f29c205 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:14.185752 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 665/1332] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:37:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:37:14.190944 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d99e45f-f387-47d6-af86-1328569bf969 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:14.191382 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d99e45f-f387-47d6-af86-1328569bf969 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:14.191674 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d99e45f-f387-47d6-af86-1328569bf969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:14.260619 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:37:14.263604 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-6d99e45f-f387-47d6-af86-1328569bf969 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:37:14.264810 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-6d99e45f-f387-47d6-af86-1328569bf969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:14.366287 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 668/1333] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:37:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:37:14.403709 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52d4b90-a634-47ce-a559-4c407f29c205 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:37:14.405244 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d52d4b90-a634-47ce-a559-4c407f29c205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:14.508878 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52d4b90-a634-47ce-a559-4c407f29c205 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:37:14.583187 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426f519c-cee8-4948-ae3d-e5267a19069d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:37:14.583861 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f519c-cee8-4948-ae3d-e5267a19069d service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:14.584146 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f519c-cee8-4948-ae3d-e5267a19069d service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:14.584423 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f519c-cee8-4948-ae3d-e5267a19069d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:14.586932 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-426f519c-cee8-4948-ae3d-e5267a19069d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:14.613611 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:37:14.694876 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426f519c-cee8-4948-ae3d-e5267a19069d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:37:14.696872 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426f519c-cee8-4948-ae3d-e5267a19069d service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:37:14.715833 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 666/1334] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:37:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:37:16.811445 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72e523-782c-43cd-ae10-6f63f60bf4a4 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:16.812058 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72e523-782c-43cd-ae10-6f63f60bf4a4 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:16.812058 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72e523-782c-43cd-ae10-6f63f60bf4a4 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:16.813417 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 669/1335] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:37:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:37:16.818561 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1652fa21-8850-4e5b-991e-9c96b2879201 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:16.819004 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1652fa21-8850-4e5b-991e-9c96b2879201 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:16.819110 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1652fa21-8850-4e5b-991e-9c96b2879201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:16.888033 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-1652fa21-8850-4e5b-991e-9c96b2879201 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:37:16.889249 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-1652fa21-8850-4e5b-991e-9c96b2879201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:16.964451 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:37:16.987262 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 667/1336] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:37:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:37:17.028133 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f72e523-782c-43cd-ae10-6f63f60bf4a4 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:37:17.029692 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-1f72e523-782c-43cd-ae10-6f63f60bf4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:17.136164 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f72e523-782c-43cd-ae10-6f63f60bf4a4 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:37:17.163560 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:37:17.211760 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e25c088-e60a-4477-83e9-30d1605c30c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:37:17.212474 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25c088-e60a-4477-83e9-30d1605c30c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:17.212758 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25c088-e60a-4477-83e9-30d1605c30c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:17.213029 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25c088-e60a-4477-83e9-30d1605c30c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:17.214688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-0e25c088-e60a-4477-83e9-30d1605c30c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:17.323116 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e25c088-e60a-4477-83e9-30d1605c30c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:37:17.324591 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e25c088-e60a-4477-83e9-30d1605c30c4 service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:37:17.342932 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 670/1337] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:37:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:37:20.233495 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c03e0-aa84-48a2-8b1c-27ca99e5af7c None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:20.234026 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c03e0-aa84-48a2-8b1c-27ca99e5af7c None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:20.234026 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c03e0-aa84-48a2-8b1c-27ca99e5af7c None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:20.235347 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 668/1338] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:37:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:37:20.240512 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf089a7-7b42-4701-8550-32dd4a1efb05 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:20.240946 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf089a7-7b42-4701-8550-32dd4a1efb05 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:20.241066 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf089a7-7b42-4701-8550-32dd4a1efb05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:20.307404 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-3cf089a7-7b42-4701-8550-32dd4a1efb05 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:37:20.308652 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-3cf089a7-7b42-4701-8550-32dd4a1efb05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:20.316780 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:37:20.407921 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 671/1339] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:37:20 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:37:20.448317 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789c03e0-aa84-48a2-8b1c-27ca99e5af7c None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:37:20.449872 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-789c03e0-aa84-48a2-8b1c-27ca99e5af7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:20.562659 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789c03e0-aa84-48a2-8b1c-27ca99e5af7c None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:37:20.635924 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b966367f-2e6d-42da-80c0-688e6287b944 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:37:20.636625 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b966367f-2e6d-42da-80c0-688e6287b944 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:20.636914 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b966367f-2e6d-42da-80c0-688e6287b944 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:20.637192 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b966367f-2e6d-42da-80c0-688e6287b944 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:20.639331 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-b966367f-2e6d-42da-80c0-688e6287b944 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:20.678111 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:37:20.752618 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b966367f-2e6d-42da-80c0-688e6287b944 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:37:20.754112 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b966367f-2e6d-42da-80c0-688e6287b944 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:37:20.772862 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 669/1340] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:37:20 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:37:25.185677 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce65b1-3d69-4cc1-97df-b3d91cdaa74f None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:25.186187 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce65b1-3d69-4cc1-97df-b3d91cdaa74f None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:25.186298 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce65b1-3d69-4cc1-97df-b3d91cdaa74f None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:25.187864 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 672/1341] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:37:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:37:25.193204 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de284f2-373c-4123-a944-e630a699053f None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:25.193641 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de284f2-373c-4123-a944-e630a699053f None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:25.193836 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de284f2-373c-4123-a944-e630a699053f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:25.263466 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-9de284f2-373c-4123-a944-e630a699053f None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:37:25.264721 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-9de284f2-373c-4123-a944-e630a699053f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:25.358620 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:37:25.370811 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 670/1342] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:37:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:37:25.410924 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ce65b1-3d69-4cc1-97df-b3d91cdaa74f None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:37:25.412533 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-b7ce65b1-3d69-4cc1-97df-b3d91cdaa74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:25.533510 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ce65b1-3d69-4cc1-97df-b3d91cdaa74f None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:37:25.572218 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:37:25.608559 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb1c090-a2cf-4a9c-a3d4-048fb4339538 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:37:25.609214 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb1c090-a2cf-4a9c-a3d4-048fb4339538 service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:25.609496 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb1c090-a2cf-4a9c-a3d4-048fb4339538 service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:25.609771 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb1c090-a2cf-4a9c-a3d4-048fb4339538 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:25.611401 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-7fb1c090-a2cf-4a9c-a3d4-048fb4339538 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:25.725368 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb1c090-a2cf-4a9c-a3d4-048fb4339538 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:37:25.726974 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb1c090-a2cf-4a9c-a3d4-048fb4339538 service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:37:25.746258 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 673/1343] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:37:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:37:28.878806 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4dae96-3e74-4d26-ba08-dbda530bfb8b None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:28.879447 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4dae96-3e74-4d26-ba08-dbda530bfb8b None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:28.879447 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4dae96-3e74-4d26-ba08-dbda530bfb8b None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:28.880761 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 671/1344] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:37:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:37:28.886585 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86175a6a-3ba6-477d-8ba1-fbd7f928c398 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:28.887029 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86175a6a-3ba6-477d-8ba1-fbd7f928c398 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:28.887148 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86175a6a-3ba6-477d-8ba1-fbd7f928c398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:28.952176 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-86175a6a-3ba6-477d-8ba1-fbd7f928c398 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:37:28.953394 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-86175a6a-3ba6-477d-8ba1-fbd7f928c398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:28.971725 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:37:29.054132 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 674/1345] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:37:28 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:37:29.092595 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4dae96-3e74-4d26-ba08-dbda530bfb8b None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:37:29.094204 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-9f4dae96-3e74-4d26-ba08-dbda530bfb8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:29.201615 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4dae96-3e74-4d26-ba08-dbda530bfb8b None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:37:29.278294 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b17ca11-ab5f-4212-b2fd-230bfdedb987 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:37:29.278948 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b17ca11-ab5f-4212-b2fd-230bfdedb987 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:29.279230 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b17ca11-ab5f-4212-b2fd-230bfdedb987 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:29.279534 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b17ca11-ab5f-4212-b2fd-230bfdedb987 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:29.281204 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-3b17ca11-ab5f-4212-b2fd-230bfdedb987 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:29.329530 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:37:29.398681 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b17ca11-ab5f-4212-b2fd-230bfdedb987 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:37:29.400157 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b17ca11-ab5f-4212-b2fd-230bfdedb987 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:37:29.420097 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 672/1346] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:37:29 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:37:34.352867 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f55cfd-40d8-4e8e-bac1-423376f99131 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:34.353423 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f55cfd-40d8-4e8e-bac1-423376f99131 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:34.353423 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f55cfd-40d8-4e8e-bac1-423376f99131 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:34.354861 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 675/1347] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:37:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:37:34.360537 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7cba3-ba45-4952-9775-e9ff00a6e4cb None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:34.361070 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7cba3-ba45-4952-9775-e9ff00a6e4cb None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:34.361070 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7cba3-ba45-4952-9775-e9ff00a6e4cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:34.434412 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-90c7cba3-ba45-4952-9775-e9ff00a6e4cb None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:37:34.435551 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-90c7cba3-ba45-4952-9775-e9ff00a6e4cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:34.539714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:37:34.542416 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 673/1348] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:37:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:37:34.582597 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f55cfd-40d8-4e8e-bac1-423376f99131 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:37:34.584184 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-56f55cfd-40d8-4e8e-bac1-423376f99131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:34.696172 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f55cfd-40d8-4e8e-bac1-423376f99131 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:37:34.748681 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:37:34.776013 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54166eb-b212-4815-8db8-ba83ff0acb01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:37:34.776680 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54166eb-b212-4815-8db8-ba83ff0acb01 service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:34.776977 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54166eb-b212-4815-8db8-ba83ff0acb01 service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:34.777220 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54166eb-b212-4815-8db8-ba83ff0acb01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:34.778926 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-a54166eb-b212-4815-8db8-ba83ff0acb01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:34.890724 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54166eb-b212-4815-8db8-ba83ff0acb01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:37:34.892209 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54166eb-b212-4815-8db8-ba83ff0acb01 service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:37:34.911722 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 676/1349] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:37:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:37:39.981411 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b178bf3-6e2e-4ed5-aedb-323e7bb4a9a2 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:39.982333 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b178bf3-6e2e-4ed5-aedb-323e7bb4a9a2 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:39.982848 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b178bf3-6e2e-4ed5-aedb-323e7bb4a9a2 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:39.984731 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 674/1350] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jul 18 16:37:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:37:39.997745 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868ef8d-7e8d-4466-93b4-9ed926f1b1c9 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:39.998275 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868ef8d-7e8d-4466-93b4-9ed926f1b1c9 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:39.998390 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868ef8d-7e8d-4466-93b4-9ed926f1b1c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:40.131372 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-d868ef8d-7e8d-4466-93b4-9ed926f1b1c9 None None] MFA Rules not processed for user `70f704a231a5474e8067a6c2dd88b355`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:37:40.132641 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-d868ef8d-7e8d-4466-93b4-9ed926f1b1c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:40.153929 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:37:40.256189 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 677/1351] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jul 18 16:37:39 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:37:44.393519 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7856510a-b3d6-404c-8314-c0d3e843a0ca None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:44.394403 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7856510a-b3d6-404c-8314-c0d3e843a0ca None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:44.400720 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7856510a-b3d6-404c-8314-c0d3e843a0ca None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:44.400720 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 675/1352] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jul 18 16:37:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:37:44.401544 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a95ae-08a2-4c67-97e7-61f366a78dc6 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:44.401974 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a95ae-08a2-4c67-97e7-61f366a78dc6 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:44.402134 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a95ae-08a2-4c67-97e7-61f366a78dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:44.485450 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-b77a95ae-08a2-4c67-97e7-61f366a78dc6 None None] MFA Rules not processed for user `70f704a231a5474e8067a6c2dd88b355`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:37:44.486752 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-b77a95ae-08a2-4c67-97e7-61f366a78dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:44.563140 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:37:44.603663 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 678/1353] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jul 18 16:37:44 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:37:45.986214 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5d84e-efe9-4df4-b00e-4cab94a503e3 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:45.986792 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5d84e-efe9-4df4-b00e-4cab94a503e3 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:45.986792 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5d84e-efe9-4df4-b00e-4cab94a503e3 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:45.988855 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 676/1354] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jul 18 16:37:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:37:45.997118 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16104b-72cc-4f29-9989-e3387cbe0626 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:45.997589 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16104b-72cc-4f29-9989-e3387cbe0626 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:45.997687 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16104b-72cc-4f29-9989-e3387cbe0626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:46.075039 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-3f16104b-72cc-4f29-9989-e3387cbe0626 None None] MFA Rules not processed for user `70f704a231a5474e8067a6c2dd88b355`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:37:46.076261 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-3f16104b-72cc-4f29-9989-e3387cbe0626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:46.186224 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 679/1355] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jul 18 16:37:45 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:37:49.748471 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db87752e-8909-426b-84c3-e3897ea77532 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:49.749326 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db87752e-8909-426b-84c3-e3897ea77532 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:49.749810 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db87752e-8909-426b-84c3-e3897ea77532 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:49.756473 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 677/1356] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:37:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:37:49.758670 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d62223-775f-4ea1-bb90-ca1229d7d3e4 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:49.759456 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d62223-775f-4ea1-bb90-ca1229d7d3e4 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:49.759788 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d62223-775f-4ea1-bb90-ca1229d7d3e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:49.804620 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:37:49.849904 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-14d62223-775f-4ea1-bb90-ca1229d7d3e4 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:37:49.851701 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-14d62223-775f-4ea1-bb90-ca1229d7d3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:49.963640 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 680/1357] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:37:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:37:50.018668 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319bd50a-a158-40ce-a7ce-89b515959903 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:50.019146 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319bd50a-a158-40ce-a7ce-89b515959903 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:50.019558 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319bd50a-a158-40ce-a7ce-89b515959903 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:50.021440 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 678/1358] 10.222.0.26 () {54 vars in 962 bytes} [Mon Jul 18 16:37:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:37:50.030957 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2bcbf9-347a-46df-b439-f10dd234546a None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:50.031581 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2bcbf9-347a-46df-b439-f10dd234546a None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:50.032071 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2bcbf9-347a-46df-b439-f10dd234546a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:50.121255 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-7c2bcbf9-347a-46df-b439-f10dd234546a None None] MFA Rules not processed for user `5b868bfa44b94ca5b470166d39f20e54`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:37:50.122762 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-7c2bcbf9-347a-46df-b439-f10dd234546a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:50.179632 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:37:50.252883 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 681/1359] 10.222.0.26 () {58 vars in 1059 bytes} [Mon Jul 18 16:37:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:37:50.261936 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319bd50a-a158-40ce-a7ce-89b515959903 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:37:50.263671 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-319bd50a-a158-40ce-a7ce-89b515959903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:50.423405 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319bd50a-a158-40ce-a7ce-89b515959903 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:37:50.509491 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:37:50.552729 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c19f839-4754-41f5-b3a1-eff68315de35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:37:50.553411 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c19f839-4754-41f5-b3a1-eff68315de35 service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:50.553714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c19f839-4754-41f5-b3a1-eff68315de35 service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:50.553991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c19f839-4754-41f5-b3a1-eff68315de35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:50.555650 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-9c19f839-4754-41f5-b3a1-eff68315de35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:50.654716 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c19f839-4754-41f5-b3a1-eff68315de35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:37:50.656498 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c19f839-4754-41f5-b3a1-eff68315de35 service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:37:50.683679 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 679/1360] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:37:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:37:52.959033 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f79282-7861-42b2-8d9a-5cb5ff37011e None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:52.959033 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f79282-7861-42b2-8d9a-5cb5ff37011e None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:52.959033 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f79282-7861-42b2-8d9a-5cb5ff37011e None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:52.959033 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 682/1361] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:37:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:37:52.962916 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb7013e-7bf2-4548-b8c4-3358974aea13 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:52.963370 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb7013e-7bf2-4548-b8c4-3358974aea13 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:52.963539 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb7013e-7bf2-4548-b8c4-3358974aea13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:53.034100 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-2bb7013e-7bf2-4548-b8c4-3358974aea13 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:37:53.035295 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-2bb7013e-7bf2-4548-b8c4-3358974aea13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:53.077029 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:37:53.144753 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 680/1362] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:37:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:37:53.191425 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f79282-7861-42b2-8d9a-5cb5ff37011e None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:37:53.192972 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-e1f79282-7861-42b2-8d9a-5cb5ff37011e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:53.364078 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f79282-7861-42b2-8d9a-5cb5ff37011e None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:37:53.375123 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:37:53.526701 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46ac0d3-c6c1-40ab-8a97-17432c82a971 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:37:53.527602 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ac0d3-c6c1-40ab-8a97-17432c82a971 service nova] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:53.528125 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ac0d3-c6c1-40ab-8a97-17432c82a971 service nova] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:53.528548 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ac0d3-c6c1-40ab-8a97-17432c82a971 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:53.530710 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-b46ac0d3-c6c1-40ab-8a97-17432c82a971 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:53.648105 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46ac0d3-c6c1-40ab-8a97-17432c82a971 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:37:53.649729 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46ac0d3-c6c1-40ab-8a97-17432c82a971 service nova] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:37:53.672162 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 683/1363] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:37:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:37:56.254558 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a213f5-1051-4562-8329-bdd26b2794c0 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:56.255106 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a213f5-1051-4562-8329-bdd26b2794c0 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:56.255106 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a213f5-1051-4562-8329-bdd26b2794c0 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:56.257084 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 681/1364] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:37:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:37:56.262702 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e101cb-baac-43ec-8450-eebe3bc29a48 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:56.263168 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e101cb-baac-43ec-8450-eebe3bc29a48 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:56.263260 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e101cb-baac-43ec-8450-eebe3bc29a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:56.278679 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:37:56.331584 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-f6e101cb-baac-43ec-8450-eebe3bc29a48 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:37:56.332841 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-f6e101cb-baac-43ec-8450-eebe3bc29a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:56.441596 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 684/1365] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:37:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:37:56.492072 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b648d2-f373-40c1-9681-99753fb30112 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:56.492389 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b648d2-f373-40c1-9681-99753fb30112 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:56.492633 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b648d2-f373-40c1-9681-99753fb30112 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:56.494199 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 682/1366] 10.222.0.26 () {54 vars in 962 bytes} [Mon Jul 18 16:37:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:37:56.501528 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d049ae-316e-4acc-9bea-4c83f34e9cbc None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:56.501822 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d049ae-316e-4acc-9bea-4c83f34e9cbc None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:56.502110 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d049ae-316e-4acc-9bea-4c83f34e9cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:56.584254 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-d9d049ae-316e-4acc-9bea-4c83f34e9cbc None None] MFA Rules not processed for user `5b868bfa44b94ca5b470166d39f20e54`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:37:56.585941 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-d9d049ae-316e-4acc-9bea-4c83f34e9cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:56.626657 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:37:56.719869 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 685/1367] 10.222.0.26 () {58 vars in 1059 bytes} [Mon Jul 18 16:37:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:37:56.728388 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b648d2-f373-40c1-9681-99753fb30112 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:37:56.729899 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-08b648d2-f373-40c1-9681-99753fb30112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:56.895932 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b648d2-f373-40c1-9681-99753fb30112 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:37:56.918626 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:37:57.026770 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cc8971-02ac-4104-868f-d699cde4588e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:37:57.027942 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc8971-02ac-4104-868f-d699cde4588e service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:57.028443 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc8971-02ac-4104-868f-d699cde4588e service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:57.028930 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc8971-02ac-4104-868f-d699cde4588e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:57.030820 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-41cc8971-02ac-4104-868f-d699cde4588e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:57.130776 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cc8971-02ac-4104-868f-d699cde4588e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:37:57.132484 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cc8971-02ac-4104-868f-d699cde4588e service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:37:57.153146 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 683/1368] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:37:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:37:58.063277 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554f1c5-61ed-44cd-acad-87df24f6910a None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:58.063861 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554f1c5-61ed-44cd-acad-87df24f6910a None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:58.063861 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554f1c5-61ed-44cd-acad-87df24f6910a None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:58.065475 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 686/1369] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:37:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:37:58.071160 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9d8b8-1822-40e6-81ff-e8602c992b50 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:58.071602 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9d8b8-1822-40e6-81ff-e8602c992b50 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:58.071814 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9d8b8-1822-40e6-81ff-e8602c992b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:58.104044 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:37:58.141790 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-bde9d8b8-1822-40e6-81ff-e8602c992b50 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:37:58.143322 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-bde9d8b8-1822-40e6-81ff-e8602c992b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:58.265972 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 684/1370] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:37:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:37:58.581946 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c826e-f239-4f22-aadc-e6d5684173a1 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:58.582225 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c826e-f239-4f22-aadc-e6d5684173a1 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:58.582500 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c826e-f239-4f22-aadc-e6d5684173a1 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:58.584037 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 687/1371] 10.222.0.26 () {54 vars in 964 bytes} [Mon Jul 18 16:37:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:37:58.589830 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5671b8-8827-495f-a318-47db296b61c0 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:58.590145 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5671b8-8827-495f-a318-47db296b61c0 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:58.590424 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5671b8-8827-495f-a318-47db296b61c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:58.672941 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-8d5671b8-8827-495f-a318-47db296b61c0 None None] MFA Rules not processed for user `6f2e55df807c43d7951fa4be67ef5e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:37:58.674164 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-8d5671b8-8827-495f-a318-47db296b61c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:58.724273 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:37:58.754230 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 685/1372] 10.222.0.26 () {58 vars in 1061 bytes} [Mon Jul 18 16:37:58 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:37:58.762563 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1c826e-f239-4f22-aadc-e6d5684173a1 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:37:58.764092 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-fe1c826e-f239-4f22-aadc-e6d5684173a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:58.890077 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1c826e-f239-4f22-aadc-e6d5684173a1 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:37:58.948505 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:37:58.988264 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b12a56-e154-440a-a064-e5378f61da52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:37:59.063813 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b12a56-e154-440a-a064-e5378f61da52 service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:59.063813 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b12a56-e154-440a-a064-e5378f61da52 service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:59.063813 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b12a56-e154-440a-a064-e5378f61da52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:59.063813 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-95b12a56-e154-440a-a064-e5378f61da52 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:59.161730 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b12a56-e154-440a-a064-e5378f61da52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:37:59.161730 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b12a56-e154-440a-a064-e5378f61da52 service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:37:59.161730 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 688/1373] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:37:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:37:59.459924 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce56d62-70fd-424c-ad94-ea2036242704 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:59.459924 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce56d62-70fd-424c-ad94-ea2036242704 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:59.459924 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce56d62-70fd-424c-ad94-ea2036242704 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:59.465134 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 686/1374] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:37:59 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:37:59.466564 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b459afeb-1522-4f13-b242-2b9a7405186d None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:59.467031 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b459afeb-1522-4f13-b242-2b9a7405186d None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:59.467188 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b459afeb-1522-4f13-b242-2b9a7405186d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:59.544624 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-b459afeb-1522-4f13-b242-2b9a7405186d None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:37:59.546572 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-b459afeb-1522-4f13-b242-2b9a7405186d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:59.578690 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:37:59.670626 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 689/1375] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:37:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:37:59.715311 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce56d62-70fd-424c-ad94-ea2036242704 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:37:59.717138 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-2ce56d62-70fd-424c-ad94-ea2036242704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:37:59.867781 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce56d62-70fd-424c-ad94-ea2036242704 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:37:59.919852 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:37:59.986452 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfee1d96-3e72-4cc5-b378-50ab0d8b0aaf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:37:59.987094 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee1d96-3e72-4cc5-b378-50ab0d8b0aaf service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:37:59.987372 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee1d96-3e72-4cc5-b378-50ab0d8b0aaf service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:37:59.987682 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee1d96-3e72-4cc5-b378-50ab0d8b0aaf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:37:59.989294 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-dfee1d96-3e72-4cc5-b378-50ab0d8b0aaf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:00.086003 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfee1d96-3e72-4cc5-b378-50ab0d8b0aaf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:00.087641 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfee1d96-3e72-4cc5-b378-50ab0d8b0aaf service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:00.106758 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 687/1376] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:37:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:38:02.560733 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227330d-e9ad-4f2a-a2c9-45af7aa6a74c None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:02.561311 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227330d-e9ad-4f2a-a2c9-45af7aa6a74c None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:02.561311 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227330d-e9ad-4f2a-a2c9-45af7aa6a74c None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:02.562945 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 690/1377] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:38:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:38:02.568236 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5688544-de8d-4a3a-a196-3f904b78dc1f None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:02.568814 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5688544-de8d-4a3a-a196-3f904b78dc1f None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:02.568896 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5688544-de8d-4a3a-a196-3f904b78dc1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:02.639774 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-d5688544-de8d-4a3a-a196-3f904b78dc1f None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:38:02.640969 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d5688544-de8d-4a3a-a196-3f904b78dc1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:02.647673 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:02.739981 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 688/1378] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:38:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:38:02.790508 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f227330d-e9ad-4f2a-a2c9-45af7aa6a74c None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:02.792052 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-f227330d-e9ad-4f2a-a2c9-45af7aa6a74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:02.972314 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f227330d-e9ad-4f2a-a2c9-45af7aa6a74c None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:02.977644 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:03.112279 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e46e311-a9df-429d-b602-fbad1e703741 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:03.113009 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e46e311-a9df-429d-b602-fbad1e703741 service nova] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:03.113388 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e46e311-a9df-429d-b602-fbad1e703741 service nova] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:03.113759 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e46e311-a9df-429d-b602-fbad1e703741 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:03.116186 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-2e46e311-a9df-429d-b602-fbad1e703741 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:03.217803 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e46e311-a9df-429d-b602-fbad1e703741 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:03.219416 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e46e311-a9df-429d-b602-fbad1e703741 service nova] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:03.244742 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 691/1379] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:38:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:38:05.519267 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a588c6-dafe-41ce-9d63-ca03fbdeef16 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:05.519880 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a588c6-dafe-41ce-9d63-ca03fbdeef16 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:05.519880 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a588c6-dafe-41ce-9d63-ca03fbdeef16 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:05.521227 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 689/1380] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:38:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:38:05.526637 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0337508c-524f-42aa-b5ee-a683b38d339c None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:05.527075 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0337508c-524f-42aa-b5ee-a683b38d339c None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:05.527366 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0337508c-524f-42aa-b5ee-a683b38d339c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:05.537623 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:05.594144 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-0337508c-524f-42aa-b5ee-a683b38d339c None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:38:05.595279 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-0337508c-524f-42aa-b5ee-a683b38d339c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:05.702283 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 692/1381] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:38:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:38:05.748667 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a588c6-dafe-41ce-9d63-ca03fbdeef16 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:05.750429 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-40a588c6-dafe-41ce-9d63-ca03fbdeef16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:05.909365 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:05.920797 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a588c6-dafe-41ce-9d63-ca03fbdeef16 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:06.043851 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754ac93c-e081-44b8-b6d2-9d8d25297e89 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:06.044603 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ac93c-e081-44b8-b6d2-9d8d25297e89 service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:06.044957 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ac93c-e081-44b8-b6d2-9d8d25297e89 service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:06.045301 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ac93c-e081-44b8-b6d2-9d8d25297e89 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:06.047040 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-754ac93c-e081-44b8-b6d2-9d8d25297e89 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:06.161115 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754ac93c-e081-44b8-b6d2-9d8d25297e89 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:06.163688 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754ac93c-e081-44b8-b6d2-9d8d25297e89 service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:06.175684 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:06.189720 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 690/1382] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:38:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:38:08.466409 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b07779-7512-4854-af3c-82bbbcf338a5 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:08.467129 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b07779-7512-4854-af3c-82bbbcf338a5 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:08.467129 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b07779-7512-4854-af3c-82bbbcf338a5 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:08.468371 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 693/1383] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:38:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:38:08.474025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e03e6-8870-40d0-bec1-03c56b1f6308 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:08.474532 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e03e6-8870-40d0-bec1-03c56b1f6308 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:08.475377 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e03e6-8870-40d0-bec1-03c56b1f6308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:08.546524 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-f52e03e6-8870-40d0-bec1-03c56b1f6308 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:38:08.547928 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f52e03e6-8870-40d0-bec1-03c56b1f6308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:08.667701 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 691/1384] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:38:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:38:08.711632 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b07779-7512-4854-af3c-82bbbcf338a5 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:08.714425 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-f9b07779-7512-4854-af3c-82bbbcf338a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:09.014905 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:09.095547 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b07779-7512-4854-af3c-82bbbcf338a5 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:09.259223 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf20d5-8d4e-4158-8e82-89de66c39ea4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:09.259922 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf20d5-8d4e-4158-8e82-89de66c39ea4 service nova] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:09.260198 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf20d5-8d4e-4158-8e82-89de66c39ea4 service nova] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:09.260479 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf20d5-8d4e-4158-8e82-89de66c39ea4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:09.262149 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-4bcf20d5-8d4e-4158-8e82-89de66c39ea4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:09.313712 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:09.355141 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcf20d5-8d4e-4158-8e82-89de66c39ea4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:09.356720 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcf20d5-8d4e-4158-8e82-89de66c39ea4 service nova] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:09.375592 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 694/1385] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:38:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 665 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:38:11.466293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e38774-0f55-4254-a6d6-4970fa1cf1f9 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:11.466832 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e38774-0f55-4254-a6d6-4970fa1cf1f9 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:11.467108 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e38774-0f55-4254-a6d6-4970fa1cf1f9 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:11.469022 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 692/1386] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:38:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:38:11.474299 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d02b4-44bb-407e-ba1f-3ff8b2dcbf70 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:11.474837 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d02b4-44bb-407e-ba1f-3ff8b2dcbf70 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:11.475008 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d02b4-44bb-407e-ba1f-3ff8b2dcbf70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:11.555286 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-4c0d02b4-44bb-407e-ba1f-3ff8b2dcbf70 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:38:11.557080 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-4c0d02b4-44bb-407e-ba1f-3ff8b2dcbf70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:11.682041 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 695/1387] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:38:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:38:11.735243 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e38774-0f55-4254-a6d6-4970fa1cf1f9 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:11.736787 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a9e38774-0f55-4254-a6d6-4970fa1cf1f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:11.832154 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:11.913110 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e38774-0f55-4254-a6d6-4970fa1cf1f9 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:12.030092 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef6460c-c261-4771-a2fc-0c3e01d67a17 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:12.030743 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6460c-c261-4771-a2fc-0c3e01d67a17 service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:12.031026 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6460c-c261-4771-a2fc-0c3e01d67a17 service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:12.031310 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6460c-c261-4771-a2fc-0c3e01d67a17 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:12.032977 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-fef6460c-c261-4771-a2fc-0c3e01d67a17 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:12.076211 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:12.134153 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef6460c-c261-4771-a2fc-0c3e01d67a17 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:12.135850 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef6460c-c261-4771-a2fc-0c3e01d67a17 service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:12.155159 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 693/1388] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:38:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:38:14.384612 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed69ff1a-6514-4db9-81b8-6a643df8c49c None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:14.385163 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed69ff1a-6514-4db9-81b8-6a643df8c49c None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:14.385285 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed69ff1a-6514-4db9-81b8-6a643df8c49c None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:14.386782 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 696/1389] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:38:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:38:14.391947 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc540ee-473d-4f05-89ea-c867b33d7432 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:14.392360 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc540ee-473d-4f05-89ea-c867b33d7432 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:14.392523 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc540ee-473d-4f05-89ea-c867b33d7432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:14.459747 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-5bc540ee-473d-4f05-89ea-c867b33d7432 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:38:14.460991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-5bc540ee-473d-4f05-89ea-c867b33d7432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:14.579520 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 694/1390] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:38:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:38:14.617258 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed69ff1a-6514-4db9-81b8-6a643df8c49c None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:14.619037 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-ed69ff1a-6514-4db9-81b8-6a643df8c49c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:14.651166 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:14.770206 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed69ff1a-6514-4db9-81b8-6a643df8c49c None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:14.892071 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346e38de-8afd-4add-b3f3-f332aa193c20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:14.892735 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e38de-8afd-4add-b3f3-f332aa193c20 service nova] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:14.893017 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e38de-8afd-4add-b3f3-f332aa193c20 service nova] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:14.893291 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e38de-8afd-4add-b3f3-f332aa193c20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:14.895028 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-346e38de-8afd-4add-b3f3-f332aa193c20 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:14.898773 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40dc136-1562-405b-8afd-a8d5aa7e1599 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:14.899142 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40dc136-1562-405b-8afd-a8d5aa7e1599 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:14.899347 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40dc136-1562-405b-8afd-a8d5aa7e1599 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:14.900903 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 695/1391] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:38:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:38:14.905868 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2652c8-54a1-4394-b19c-f116e59ef492 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:14.906126 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2652c8-54a1-4394-b19c-f116e59ef492 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:14.906408 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2652c8-54a1-4394-b19c-f116e59ef492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:14.920634 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:14.924177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:14.973056 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-4f2652c8-54a1-4394-b19c-f116e59ef492 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:38:14.974269 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-4f2652c8-54a1-4394-b19c-f116e59ef492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:14.987868 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346e38de-8afd-4add-b3f3-f332aa193c20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:14.989507 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346e38de-8afd-4add-b3f3-f332aa193c20 service nova] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:15.008064 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 697/1392] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:38:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:38:15.074538 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 696/1393] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:38:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:38:15.136948 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a04092-5bbb-4db7-84af-83c39c538600 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:15.138275 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a04092-5bbb-4db7-84af-83c39c538600 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:15.138275 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a04092-5bbb-4db7-84af-83c39c538600 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:15.138938 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 698/1394] 10.222.0.26 () {54 vars in 961 bytes} [Mon Jul 18 16:38:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:38:15.144212 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd865322-ac1e-4bc7-8bbb-c81a44c5c2a5 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:15.144504 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd865322-ac1e-4bc7-8bbb-c81a44c5c2a5 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:15.144811 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd865322-ac1e-4bc7-8bbb-c81a44c5c2a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:15.222914 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-cd865322-ac1e-4bc7-8bbb-c81a44c5c2a5 None None] MFA Rules not processed for user `0887908bcb0d4072bf6acb4cbaf1af83`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:38:15.224195 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-cd865322-ac1e-4bc7-8bbb-c81a44c5c2a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:15.261765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:15.309932 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 697/1395] 10.222.0.26 () {58 vars in 1058 bytes} [Mon Jul 18 16:38:15 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:38:15.317840 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a04092-5bbb-4db7-84af-83c39c538600 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:15.320270 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-a3a04092-5bbb-4db7-84af-83c39c538600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:15.435504 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a04092-5bbb-4db7-84af-83c39c538600 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:15.450133 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:15.514975 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d51774-a8b3-46c2-85f6-3d52bb954b29 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0887908bcb0d4072bf6acb4cbaf1af83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:15.515679 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d51774-a8b3-46c2-85f6-3d52bb954b29 service glance] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:15.515968 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d51774-a8b3-46c2-85f6-3d52bb954b29 service glance] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:15.516241 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d51774-a8b3-46c2-85f6-3d52bb954b29 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:15.517883 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-11d51774-a8b3-46c2-85f6-3d52bb954b29 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:15.630428 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d51774-a8b3-46c2-85f6-3d52bb954b29 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:15.633105 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d51774-a8b3-46c2-85f6-3d52bb954b29 service glance] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:15.652459 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 699/1396] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:38:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:38:16.564951 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c3b34-1b15-4509-86f7-432489b0885c None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:16.565432 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c3b34-1b15-4509-86f7-432489b0885c None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:16.565682 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c3b34-1b15-4509-86f7-432489b0885c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:16.632969 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-0f8c3b34-1b15-4509-86f7-432489b0885c None None] MFA Rules not processed for user `da799b0044f54da6835aa06e43ce1a6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:38:16.634188 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-0f8c3b34-1b15-4509-86f7-432489b0885c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:16.725909 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 698/1397] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jul 18 16:38:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:38:16.773961 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51766159-ebec-42b8-aed1-e46441c2556c None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:16.774491 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51766159-ebec-42b8-aed1-e46441c2556c None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:16.774793 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51766159-ebec-42b8-aed1-e46441c2556c None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:16.776606 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 700/1398] 10.222.0.26 () {54 vars in 943 bytes} [Mon Jul 18 16:38:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:38:16.783485 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c443904-24b0-402c-a431-4a858e93d3e2 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:16.783801 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c443904-24b0-402c-a431-4a858e93d3e2 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:16.784083 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c443904-24b0-402c-a431-4a858e93d3e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:16.848728 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:16.868333 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-5c443904-24b0-402c-a431-4a858e93d3e2 None None] MFA Rules not processed for user `43854d7a6df2466eb7976fabe6011767`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:38:16.869542 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-5c443904-24b0-402c-a431-4a858e93d3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:16.983553 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 699/1399] 10.222.0.26 () {58 vars in 1040 bytes} [Mon Jul 18 16:38:16 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:38:16.994409 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51766159-ebec-42b8-aed1-e46441c2556c None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:16.996088 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-51766159-ebec-42b8-aed1-e46441c2556c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:17.045334 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:17.149990 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51766159-ebec-42b8-aed1-e46441c2556c None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:17.264388 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f297444a-4bec-4c2c-b4cd-b726749d5033 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43854d7a6df2466eb7976fabe6011767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:17.265049 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297444a-4bec-4c2c-b4cd-b726749d5033 service swift] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:17.265333 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297444a-4bec-4c2c-b4cd-b726749d5033 service swift] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:17.265606 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297444a-4bec-4c2c-b4cd-b726749d5033 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:17.267292 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-f297444a-4bec-4c2c-b4cd-b726749d5033 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:17.299036 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:17.306997 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f81f4-86ca-4f34-9bb8-2d32aa812b23 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:17.307291 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f81f4-86ca-4f34-9bb8-2d32aa812b23 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:17.307603 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f81f4-86ca-4f34-9bb8-2d32aa812b23 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:17.309221 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 700/1400] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:38:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:38:17.314135 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631763c-f522-4ead-a90c-f55673627977 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:17.314425 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631763c-f522-4ead-a90c-f55673627977 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:17.314705 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631763c-f522-4ead-a90c-f55673627977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:17.356119 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f297444a-4bec-4c2c-b4cd-b726749d5033 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:17.357724 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f297444a-4bec-4c2c-b4cd-b726749d5033 service swift] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:17.366698 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 701/1401] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jul 18 16:38:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 16:38:17.382399 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-9631763c-f522-4ead-a90c-f55673627977 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:38:17.383646 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-9631763c-f522-4ead-a90c-f55673627977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:17.442468 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:17.484498 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 701/1402] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:38:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:38:17.529817 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f297444a-4bec-4c2c-b4cd-b726749d5033 service swift] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:17.531039 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-f297444a-4bec-4c2c-b4cd-b726749d5033 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:17.625488 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f297444a-4bec-4c2c-b4cd-b726749d5033 service swift] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:17.698057 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:17.742392 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58a220a-bfc5-40d7-bb06-41204c612b8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:17.743034 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58a220a-bfc5-40d7-bb06-41204c612b8d service nova] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:17.743310 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58a220a-bfc5-40d7-bb06-41204c612b8d service nova] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:17.743613 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58a220a-bfc5-40d7-bb06-41204c612b8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:17.745241 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-c58a220a-bfc5-40d7-bb06-41204c612b8d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:17.848021 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58a220a-bfc5-40d7-bb06-41204c612b8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:17.849664 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58a220a-bfc5-40d7-bb06-41204c612b8d service nova] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:17.877289 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 702/1403] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:38:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:38:20.310665 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e024a-e9bd-46b0-8ee1-c8f9b2315f6c None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:20.311245 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e024a-e9bd-46b0-8ee1-c8f9b2315f6c None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:20.311245 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e024a-e9bd-46b0-8ee1-c8f9b2315f6c None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:20.313068 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 702/1404] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:38:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:38:20.318656 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70ff1b-10fb-457f-af2a-d3b47cdd3db8 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:20.319345 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70ff1b-10fb-457f-af2a-d3b47cdd3db8 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:20.319345 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70ff1b-10fb-457f-af2a-d3b47cdd3db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:20.401556 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-1b70ff1b-10fb-457f-af2a-d3b47cdd3db8 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:38:20.403192 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-1b70ff1b-10fb-457f-af2a-d3b47cdd3db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:20.441755 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:20.523533 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 703/1405] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:38:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:38:20.566291 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e024a-e9bd-46b0-8ee1-c8f9b2315f6c None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:20.568675 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-105e024a-e9bd-46b0-8ee1-c8f9b2315f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:20.753555 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e024a-e9bd-46b0-8ee1-c8f9b2315f6c None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:20.799752 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:20.891252 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83704a10-4a6e-4309-8251-acabf3b0d33b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:20.891956 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83704a10-4a6e-4309-8251-acabf3b0d33b service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:20.892248 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83704a10-4a6e-4309-8251-acabf3b0d33b service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:20.892529 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83704a10-4a6e-4309-8251-acabf3b0d33b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:20.894289 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-83704a10-4a6e-4309-8251-acabf3b0d33b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:20.998847 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83704a10-4a6e-4309-8251-acabf3b0d33b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:21.000498 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83704a10-4a6e-4309-8251-acabf3b0d33b service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:21.020749 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 703/1406] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:38:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:38:23.496411 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5fe84-42cd-4c82-b642-47beb856ff86 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:23.496977 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5fe84-42cd-4c82-b642-47beb856ff86 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:23.496977 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5fe84-42cd-4c82-b642-47beb856ff86 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:23.498351 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 704/1407] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:38:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:38:23.503662 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01caca9-8cd2-4875-a00c-48f44aa328b2 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:23.504180 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01caca9-8cd2-4875-a00c-48f44aa328b2 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:23.504496 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01caca9-8cd2-4875-a00c-48f44aa328b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:23.581754 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:23.590303 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-c01caca9-8cd2-4875-a00c-48f44aa328b2 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:38:23.591718 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-c01caca9-8cd2-4875-a00c-48f44aa328b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:23.715011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 704/1408] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:38:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:38:23.760683 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5fe84-42cd-4c82-b642-47beb856ff86 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:23.762287 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-dbc5fe84-42cd-4c82-b642-47beb856ff86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:23.929864 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5fe84-42cd-4c82-b642-47beb856ff86 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:23.949474 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:24.059695 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909ce32c-e32f-46c4-82b0-6448a93e8e58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:24.060283 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ce32c-e32f-46c4-82b0-6448a93e8e58 service nova] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:24.060566 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ce32c-e32f-46c4-82b0-6448a93e8e58 service nova] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:24.060839 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ce32c-e32f-46c4-82b0-6448a93e8e58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:24.062518 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-909ce32c-e32f-46c4-82b0-6448a93e8e58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:24.171459 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909ce32c-e32f-46c4-82b0-6448a93e8e58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:24.173135 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909ce32c-e32f-46c4-82b0-6448a93e8e58 service nova] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:24.194300 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 705/1409] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:38:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:38:33.713162 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=44 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:34.191394 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=90 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:42.503818 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75beeda6-77de-4e53-b411-c2cf22c05426 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:42.504395 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75beeda6-77de-4e53-b411-c2cf22c05426 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:42.504395 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75beeda6-77de-4e53-b411-c2cf22c05426 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:42.505759 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 705/1410] 10.222.0.26 () {54 vars in 918 bytes} [Mon Jul 18 16:38:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:38:42.511265 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d97d4-f644-492a-b6d1-691894044441 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:42.511744 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d97d4-f644-492a-b6d1-691894044441 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:42.511904 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d97d4-f644-492a-b6d1-691894044441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:42.589179 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-314d97d4-f644-492a-b6d1-691894044441 None None] MFA Rules not processed for user `70f704a231a5474e8067a6c2dd88b355`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:38:42.590393 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-314d97d4-f644-492a-b6d1-691894044441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:42.702006 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 706/1411] 10.222.0.26 () {58 vars in 1015 bytes} [Mon Jul 18 16:38:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:38:42.717286 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d56e7-6751-45e0-86c0-84c470445d9e None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:42.717646 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d56e7-6751-45e0-86c0-84c470445d9e None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:42.717854 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d56e7-6751-45e0-86c0-84c470445d9e None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:42.719218 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 706/1412] 10.222.0.26 () {54 vars in 918 bytes} [Mon Jul 18 16:38:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:38:42.724009 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517dcaf8-2e52-4914-8ca0-38400b58596c None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:42.724328 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517dcaf8-2e52-4914-8ca0-38400b58596c None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:42.724599 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517dcaf8-2e52-4914-8ca0-38400b58596c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:42.795286 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:42.797730 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-517dcaf8-2e52-4914-8ca0-38400b58596c None None] MFA Rules not processed for user `5b868bfa44b94ca5b470166d39f20e54`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:38:42.798934 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-517dcaf8-2e52-4914-8ca0-38400b58596c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:42.917870 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 707/1413] 10.222.0.26 () {58 vars in 1015 bytes} [Mon Jul 18 16:38:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:38:45.410596 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b62b28-7e7e-4f11-8341-b13221b49fb0 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:45.411166 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b62b28-7e7e-4f11-8341-b13221b49fb0 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:45.411166 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b62b28-7e7e-4f11-8341-b13221b49fb0 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:45.413332 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 707/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 16:38:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:38:45.418725 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b863550-0ad0-4f6b-b093-7d41a61ad6d6 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:45.419205 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b863550-0ad0-4f6b-b093-7d41a61ad6d6 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:45.419303 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b863550-0ad0-4f6b-b093-7d41a61ad6d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:45.484103 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-2b863550-0ad0-4f6b-b093-7d41a61ad6d6 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:38:45.485223 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-2b863550-0ad0-4f6b-b093-7d41a61ad6d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:45.540506 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:45.582720 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 708/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 16:38:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:38:45.626556 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b62b28-7e7e-4f11-8341-b13221b49fb0 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:45.628084 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e3b62b28-7e7e-4f11-8341-b13221b49fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:45.779211 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b62b28-7e7e-4f11-8341-b13221b49fb0 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:45.854926 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:45.896257 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09b5996-2c78-43f2-b435-ae499669bcc9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:45.897098 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b5996-2c78-43f2-b435-ae499669bcc9 service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:45.897388 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b5996-2c78-43f2-b435-ae499669bcc9 service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:45.897664 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b5996-2c78-43f2-b435-ae499669bcc9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:45.899346 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-b09b5996-2c78-43f2-b435-ae499669bcc9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:45.992280 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09b5996-2c78-43f2-b435-ae499669bcc9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:45.994340 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09b5996-2c78-43f2-b435-ae499669bcc9 service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:46.014694 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 708/1416] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:38:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:38:46.938761 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfac4fb-5b9a-4042-a623-0d35da845831 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:46.939372 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfac4fb-5b9a-4042-a623-0d35da845831 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:46.939372 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfac4fb-5b9a-4042-a623-0d35da845831 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:46.940726 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 709/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 16:38:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:38:46.946407 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b800b-d1f7-4b2e-b455-7a43c14766eb None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:46.946884 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b800b-d1f7-4b2e-b455-7a43c14766eb None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:46.946990 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b800b-d1f7-4b2e-b455-7a43c14766eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:47.021787 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-ea2b800b-d1f7-4b2e-b455-7a43c14766eb None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:38:47.023003 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-ea2b800b-d1f7-4b2e-b455-7a43c14766eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:47.050149 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:47.121544 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 709/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 16:38:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:38:47.273641 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfac4fb-5b9a-4042-a623-0d35da845831 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:47.275159 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-5dfac4fb-5b9a-4042-a623-0d35da845831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:47.421377 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfac4fb-5b9a-4042-a623-0d35da845831 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:47.462840 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:47.537036 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521ff9d1-6c20-4d3b-8b7e-b7f64e3bf50d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:47.537694 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ff9d1-6c20-4d3b-8b7e-b7f64e3bf50d service nova] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:47.537978 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ff9d1-6c20-4d3b-8b7e-b7f64e3bf50d service nova] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:47.538258 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ff9d1-6c20-4d3b-8b7e-b7f64e3bf50d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:47.539950 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-521ff9d1-6c20-4d3b-8b7e-b7f64e3bf50d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:47.630670 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521ff9d1-6c20-4d3b-8b7e-b7f64e3bf50d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:47.632292 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521ff9d1-6c20-4d3b-8b7e-b7f64e3bf50d service nova] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:47.650737 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 710/1419] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:38:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:38:48.555901 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6456b-b492-40a4-8b9d-2740dcfb567f None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:48.556444 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6456b-b492-40a4-8b9d-2740dcfb567f None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:48.556444 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6456b-b492-40a4-8b9d-2740dcfb567f None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:48.557858 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 710/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 16:38:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:38:48.562899 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a8df91-cf99-4ce7-97c3-3b5d01c46bcd None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:48.563345 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a8df91-cf99-4ce7-97c3-3b5d01c46bcd None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:48.563463 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a8df91-cf99-4ce7-97c3-3b5d01c46bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:48.620537 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:48.642493 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-a1a8df91-cf99-4ce7-97c3-3b5d01c46bcd None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:38:48.643784 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-a1a8df91-cf99-4ce7-97c3-3b5d01c46bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:48.743610 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 711/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 16:38:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:38:48.898104 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d6456b-b492-40a4-8b9d-2740dcfb567f None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:48.899704 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f3d6456b-b492-40a4-8b9d-2740dcfb567f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:49.048415 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d6456b-b492-40a4-8b9d-2740dcfb567f None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:49.061339 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:49.165283 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d7f53-01ed-4f22-a29e-d8903745a49c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:49.165862 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d7f53-01ed-4f22-a29e-d8903745a49c service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:49.166144 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d7f53-01ed-4f22-a29e-d8903745a49c service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:49.166417 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d7f53-01ed-4f22-a29e-d8903745a49c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:49.168229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-fb1d7f53-01ed-4f22-a29e-d8903745a49c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:49.263140 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1d7f53-01ed-4f22-a29e-d8903745a49c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:49.264778 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1d7f53-01ed-4f22-a29e-d8903745a49c service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:49.283458 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 711/1422] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:38:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:38:50.170052 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd44a7cf-c078-409b-8eab-84e73af29258 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:50.170571 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd44a7cf-c078-409b-8eab-84e73af29258 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:50.170571 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd44a7cf-c078-409b-8eab-84e73af29258 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:50.171947 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 712/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 16:38:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:38:50.178038 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7b5d7-c924-4bea-81a6-eb67311a82dc None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:50.178516 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7b5d7-c924-4bea-81a6-eb67311a82dc None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:50.178629 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7b5d7-c924-4bea-81a6-eb67311a82dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:50.197826 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:50.260113 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-a5c7b5d7-c924-4bea-81a6-eb67311a82dc None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:38:50.261356 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a5c7b5d7-c924-4bea-81a6-eb67311a82dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:50.366185 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 712/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 16:38:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:38:50.521865 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd44a7cf-c078-409b-8eab-84e73af29258 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:50.523389 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-bd44a7cf-c078-409b-8eab-84e73af29258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:50.635020 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:50.673077 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd44a7cf-c078-409b-8eab-84e73af29258 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:50.789068 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15bba3a-1496-4e9c-a0f0-cc926cb94977 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:50.789640 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15bba3a-1496-4e9c-a0f0-cc926cb94977 service nova] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:50.790017 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15bba3a-1496-4e9c-a0f0-cc926cb94977 service nova] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:50.790348 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15bba3a-1496-4e9c-a0f0-cc926cb94977 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:50.792066 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-c15bba3a-1496-4e9c-a0f0-cc926cb94977 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:50.873671 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:50.887550 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15bba3a-1496-4e9c-a0f0-cc926cb94977 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:50.889145 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15bba3a-1496-4e9c-a0f0-cc926cb94977 service nova] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:50.907417 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 713/1425] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:38:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:38:51.790210 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed58d48-98a5-4d5e-89eb-28aeef2a1e6e None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:51.790792 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed58d48-98a5-4d5e-89eb-28aeef2a1e6e None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:51.790792 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed58d48-98a5-4d5e-89eb-28aeef2a1e6e None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:51.792167 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 713/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 16:38:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:38:51.797632 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e264be-0597-4cba-b88b-5bd8258fb9ca None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:51.798074 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e264be-0597-4cba-b88b-5bd8258fb9ca None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:51.798209 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e264be-0597-4cba-b88b-5bd8258fb9ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:51.878045 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-55e264be-0597-4cba-b88b-5bd8258fb9ca None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:38:51.879175 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-55e264be-0597-4cba-b88b-5bd8258fb9ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:51.975793 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 714/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 16:38:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:38:52.126627 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed58d48-98a5-4d5e-89eb-28aeef2a1e6e None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:52.128246 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-7ed58d48-98a5-4d5e-89eb-28aeef2a1e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:52.214116 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:52.278755 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed58d48-98a5-4d5e-89eb-28aeef2a1e6e None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:52.395228 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0890f30e-fe04-4d93-a39b-6931e5e3d6b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:52.396592 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0890f30e-fe04-4d93-a39b-6931e5e3d6b0 service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:52.396882 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0890f30e-fe04-4d93-a39b-6931e5e3d6b0 service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:52.397162 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0890f30e-fe04-4d93-a39b-6931e5e3d6b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:52.398813 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-0890f30e-fe04-4d93-a39b-6931e5e3d6b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:52.448710 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:52.490702 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0890f30e-fe04-4d93-a39b-6931e5e3d6b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:52.492363 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0890f30e-fe04-4d93-a39b-6931e5e3d6b0 service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:52.511002 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 714/1428] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:38:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:38:53.420493 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a5d9b-d2bf-486b-aed8-db8bf84f58f9 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:53.421094 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a5d9b-d2bf-486b-aed8-db8bf84f58f9 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:53.421094 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a5d9b-d2bf-486b-aed8-db8bf84f58f9 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:53.422429 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 715/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 16:38:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:38:53.428075 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b56146c-da97-4b05-a5f3-a99d2039cf30 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:53.428679 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b56146c-da97-4b05-a5f3-a99d2039cf30 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:53.428796 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b56146c-da97-4b05-a5f3-a99d2039cf30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:53.503993 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-9b56146c-da97-4b05-a5f3-a99d2039cf30 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:38:53.505887 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-9b56146c-da97-4b05-a5f3-a99d2039cf30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:53.603230 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 715/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 16:38:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:38:53.757174 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4a5d9b-d2bf-486b-aed8-db8bf84f58f9 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:53.758623 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-8a4a5d9b-d2bf-486b-aed8-db8bf84f58f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:53.808935 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:53.911772 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4a5d9b-d2bf-486b-aed8-db8bf84f58f9 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:54.026155 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900760f2-e43f-45f4-88cb-339eb080e06e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:54.026727 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900760f2-e43f-45f4-88cb-339eb080e06e service nova] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:54.027010 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900760f2-e43f-45f4-88cb-339eb080e06e service nova] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:54.027287 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900760f2-e43f-45f4-88cb-339eb080e06e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:54.028959 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-900760f2-e43f-45f4-88cb-339eb080e06e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:54.036685 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:54.120020 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900760f2-e43f-45f4-88cb-339eb080e06e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:54.121614 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900760f2-e43f-45f4-88cb-339eb080e06e service nova] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:54.139248 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 716/1431] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:38:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:38:55.040485 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101978d4-2f0d-4b39-b85f-d89e5b78d0b4 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:55.041058 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101978d4-2f0d-4b39-b85f-d89e5b78d0b4 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:55.041058 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101978d4-2f0d-4b39-b85f-d89e5b78d0b4 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:55.042451 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 716/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 16:38:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:38:55.047437 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87afbbe6-4ac9-49b3-919f-167726cd3fd3 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:55.047874 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87afbbe6-4ac9-49b3-919f-167726cd3fd3 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:55.048038 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87afbbe6-4ac9-49b3-919f-167726cd3fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:55.123545 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-87afbbe6-4ac9-49b3-919f-167726cd3fd3 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:38:55.124696 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-87afbbe6-4ac9-49b3-919f-167726cd3fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:55.193632 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:55.222562 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 717/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 16:38:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:38:55.377629 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101978d4-2f0d-4b39-b85f-d89e5b78d0b4 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:55.379066 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-101978d4-2f0d-4b39-b85f-d89e5b78d0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:55.397736 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:55.525766 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101978d4-2f0d-4b39-b85f-d89e5b78d0b4 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:55.613318 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:55.640191 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04994b7e-75d6-469d-84e7-8f9fc704b1ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:55.640768 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04994b7e-75d6-469d-84e7-8f9fc704b1ef service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:55.641052 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04994b7e-75d6-469d-84e7-8f9fc704b1ef service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:55.641328 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04994b7e-75d6-469d-84e7-8f9fc704b1ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:55.642984 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-04994b7e-75d6-469d-84e7-8f9fc704b1ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:55.731830 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04994b7e-75d6-469d-84e7-8f9fc704b1ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:55.733472 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04994b7e-75d6-469d-84e7-8f9fc704b1ef service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:55.752511 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 717/1434] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:38:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:38:56.664172 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad6570d-b3cd-4e76-be38-ba461e9c44d6 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:56.664916 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad6570d-b3cd-4e76-be38-ba461e9c44d6 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:56.664916 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad6570d-b3cd-4e76-be38-ba461e9c44d6 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:56.666114 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 718/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 16:38:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:38:56.671196 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19ce695-af4a-4dbe-86ac-a1ddd96d2a05 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:56.671648 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19ce695-af4a-4dbe-86ac-a1ddd96d2a05 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:56.671772 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19ce695-af4a-4dbe-86ac-a1ddd96d2a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:56.750055 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-a19ce695-af4a-4dbe-86ac-a1ddd96d2a05 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:38:56.751268 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a19ce695-af4a-4dbe-86ac-a1ddd96d2a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:56.794700 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:56.853870 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 718/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 16:38:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:38:57.008974 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad6570d-b3cd-4e76-be38-ba461e9c44d6 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:57.010484 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-7ad6570d-b3cd-4e76-be38-ba461e9c44d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:57.158834 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad6570d-b3cd-4e76-be38-ba461e9c44d6 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:57.214159 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:57.276979 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212d8714-5881-42aa-aa0a-a145ea2cbebb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:57.277556 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212d8714-5881-42aa-aa0a-a145ea2cbebb service nova] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:57.277840 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212d8714-5881-42aa-aa0a-a145ea2cbebb service nova] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:57.278115 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212d8714-5881-42aa-aa0a-a145ea2cbebb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:57.279816 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-212d8714-5881-42aa-aa0a-a145ea2cbebb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:57.371069 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212d8714-5881-42aa-aa0a-a145ea2cbebb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:57.372710 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212d8714-5881-42aa-aa0a-a145ea2cbebb service nova] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:57.391122 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 719/1437] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:38:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:38:58.319611 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c8ee7-36bd-44e2-aa61-51e06bb0a28f None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:58.320183 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c8ee7-36bd-44e2-aa61-51e06bb0a28f None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:58.320183 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c8ee7-36bd-44e2-aa61-51e06bb0a28f None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:58.321557 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 719/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 16:38:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:38:58.326623 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe02d57-732d-42a4-828c-68c35deb9a0c None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:58.327062 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe02d57-732d-42a4-828c-68c35deb9a0c None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:58.327187 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe02d57-732d-42a4-828c-68c35deb9a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:58.403017 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-ebe02d57-732d-42a4-828c-68c35deb9a0c None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:38:58.404668 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-ebe02d57-732d-42a4-828c-68c35deb9a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:58.406734 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:58.505100 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 720/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 16:38:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:38:58.658167 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169c8ee7-36bd-44e2-aa61-51e06bb0a28f None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:38:58.659752 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-169c8ee7-36bd-44e2-aa61-51e06bb0a28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:58.808087 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169c8ee7-36bd-44e2-aa61-51e06bb0a28f None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:38:58.837174 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:38:58.924354 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb864c9-64e0-4639-975e-18e5b52cabff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:38:58.924927 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb864c9-64e0-4639-975e-18e5b52cabff service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:38:58.925213 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb864c9-64e0-4639-975e-18e5b52cabff service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:38:58.925492 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb864c9-64e0-4639-975e-18e5b52cabff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:38:58.927783 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-6eb864c9-64e0-4639-975e-18e5b52cabff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:38:59.018084 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb864c9-64e0-4639-975e-18e5b52cabff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:38:59.019760 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb864c9-64e0-4639-975e-18e5b52cabff service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:38:59.038689 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 720/1440] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:38:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:39:01.045129 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ceb64-a298-446a-84f9-9ac6de7db803 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:01.045710 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ceb64-a298-446a-84f9-9ac6de7db803 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:01.045710 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ceb64-a298-446a-84f9-9ac6de7db803 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:01.047071 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 721/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 16:39:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:39:01.054209 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a678557-2863-46b2-b4e4-f1d2d2ca5b75 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:01.054676 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a678557-2863-46b2-b4e4-f1d2d2ca5b75 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:01.054981 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a678557-2863-46b2-b4e4-f1d2d2ca5b75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:01.099745 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:01.131305 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-3a678557-2863-46b2-b4e4-f1d2d2ca5b75 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:01.132585 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-3a678557-2863-46b2-b4e4-f1d2d2ca5b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:01.256040 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 721/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 16:39:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:01.303037 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967ceb64-a298-446a-84f9-9ac6de7db803 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:01.304227 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-967ceb64-a298-446a-84f9-9ac6de7db803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:01.410077 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967ceb64-a298-446a-84f9-9ac6de7db803 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:01.443391 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:01.485656 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f57c03c-25f3-4028-97f5-757e36ed4111 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:01.486315 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57c03c-25f3-4028-97f5-757e36ed4111 service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:01.486599 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57c03c-25f3-4028-97f5-757e36ed4111 service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:01.486876 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57c03c-25f3-4028-97f5-757e36ed4111 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:01.488567 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-9f57c03c-25f3-4028-97f5-757e36ed4111 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:01.600154 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f57c03c-25f3-4028-97f5-757e36ed4111 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:01.601614 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f57c03c-25f3-4028-97f5-757e36ed4111 service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:01.621586 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 722/1443] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:39:03.832947 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cf7ed0-93a9-4e17-bcc5-84c95424ac2d None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:03.833555 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cf7ed0-93a9-4e17-bcc5-84c95424ac2d None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:03.833555 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cf7ed0-93a9-4e17-bcc5-84c95424ac2d None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:03.834850 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 722/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 16:39:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:39:03.840340 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc89b3b-3f55-44bd-b11f-19bf75ee0842 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:03.840793 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc89b3b-3f55-44bd-b11f-19bf75ee0842 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:03.840900 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc89b3b-3f55-44bd-b11f-19bf75ee0842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:03.910636 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-0bc89b3b-3f55-44bd-b11f-19bf75ee0842 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:03.912050 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-0bc89b3b-3f55-44bd-b11f-19bf75ee0842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:03.924821 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:04.011051 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 723/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 16:39:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:39:04.050330 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cf7ed0-93a9-4e17-bcc5-84c95424ac2d None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:04.052938 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-64cf7ed0-93a9-4e17-bcc5-84c95424ac2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:04.162165 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cf7ed0-93a9-4e17-bcc5-84c95424ac2d None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:04.174382 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:04.239758 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543ed158-52fb-4189-aa39-11756fe015b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:04.240414 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ed158-52fb-4189-aa39-11756fe015b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:04.240701 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ed158-52fb-4189-aa39-11756fe015b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:04.240981 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ed158-52fb-4189-aa39-11756fe015b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:04.242682 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-543ed158-52fb-4189-aa39-11756fe015b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:04.355785 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543ed158-52fb-4189-aa39-11756fe015b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:04.357233 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543ed158-52fb-4189-aa39-11756fe015b2 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:04.375539 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 723/1446] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:39:06.851327 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ceb88-74d3-4b85-b1e7-6f00d67e4393 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:06.851859 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ceb88-74d3-4b85-b1e7-6f00d67e4393 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:06.851989 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ceb88-74d3-4b85-b1e7-6f00d67e4393 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:06.853293 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 724/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 16:39:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:39:06.858714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15277970-eefd-4d34-abe0-6ec1005d4cbd None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:06.859109 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15277970-eefd-4d34-abe0-6ec1005d4cbd None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:06.859277 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15277970-eefd-4d34-abe0-6ec1005d4cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:06.909304 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:06.927645 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-15277970-eefd-4d34-abe0-6ec1005d4cbd None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:06.928861 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-15277970-eefd-4d34-abe0-6ec1005d4cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:07.026611 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 724/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 16:39:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:07.066870 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4ceb88-74d3-4b85-b1e7-6f00d67e4393 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:07.069229 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-8f4ceb88-74d3-4b85-b1e7-6f00d67e4393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:07.173689 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4ceb88-74d3-4b85-b1e7-6f00d67e4393 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:07.216006 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:07.252404 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388042ca-840b-418f-8248-fc532e58d027 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:07.253070 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388042ca-840b-418f-8248-fc532e58d027 service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:07.253554 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388042ca-840b-418f-8248-fc532e58d027 service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:07.254055 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388042ca-840b-418f-8248-fc532e58d027 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:07.257175 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-388042ca-840b-418f-8248-fc532e58d027 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:07.366094 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388042ca-840b-418f-8248-fc532e58d027 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:07.367574 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388042ca-840b-418f-8248-fc532e58d027 service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:07.386058 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 725/1449] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:39:09.637290 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e1b322-d9fe-491f-b0c1-2a325887607c None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:09.637837 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e1b322-d9fe-491f-b0c1-2a325887607c None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:09.637837 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e1b322-d9fe-491f-b0c1-2a325887607c None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:09.639367 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 725/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 16:39:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:39:09.644750 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1157dae-c3c6-4840-b128-055d211a80bd None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:09.645830 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1157dae-c3c6-4840-b128-055d211a80bd None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:09.645966 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1157dae-c3c6-4840-b128-055d211a80bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:09.712266 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-a1157dae-c3c6-4840-b128-055d211a80bd None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:09.713493 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-a1157dae-c3c6-4840-b128-055d211a80bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:09.735702 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:09.813111 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 726/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 16:39:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:09.854914 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e1b322-d9fe-491f-b0c1-2a325887607c None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:09.856506 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-59e1b322-d9fe-491f-b0c1-2a325887607c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:09.965313 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e1b322-d9fe-491f-b0c1-2a325887607c None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:09.985664 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:10.040329 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645f61ee-0f79-4369-a05b-d6fdf2821a5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:10.040984 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645f61ee-0f79-4369-a05b-d6fdf2821a5b service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:10.041269 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645f61ee-0f79-4369-a05b-d6fdf2821a5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:10.041550 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645f61ee-0f79-4369-a05b-d6fdf2821a5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:10.043195 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-645f61ee-0f79-4369-a05b-d6fdf2821a5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:10.153341 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645f61ee-0f79-4369-a05b-d6fdf2821a5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:10.154785 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645f61ee-0f79-4369-a05b-d6fdf2821a5b service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:10.173492 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 726/1452] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:39:12.471754 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3270859-0604-4c89-8993-69e63b5c0b3a None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:12.472348 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3270859-0604-4c89-8993-69e63b5c0b3a None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:12.472348 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3270859-0604-4c89-8993-69e63b5c0b3a None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:12.473699 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 727/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 16:39:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:39:12.479485 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3ca29-d24a-455b-9e92-001d473e07e2 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:12.480018 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3ca29-d24a-455b-9e92-001d473e07e2 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:12.480116 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3ca29-d24a-455b-9e92-001d473e07e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:12.547038 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:12.549693 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-fbc3ca29-d24a-455b-9e92-001d473e07e2 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:12.550942 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-fbc3ca29-d24a-455b-9e92-001d473e07e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:12.653511 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 727/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 16:39:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:12.691568 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3270859-0604-4c89-8993-69e63b5c0b3a None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:12.693850 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-a3270859-0604-4c89-8993-69e63b5c0b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:12.803267 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3270859-0604-4c89-8993-69e63b5c0b3a None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:12.853851 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:12.879150 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe164df2-18d5-4b40-bead-8cfd5230a783 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:12.880114 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe164df2-18d5-4b40-bead-8cfd5230a783 service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:12.880408 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe164df2-18d5-4b40-bead-8cfd5230a783 service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:12.880683 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe164df2-18d5-4b40-bead-8cfd5230a783 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:12.882390 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-fe164df2-18d5-4b40-bead-8cfd5230a783 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:12.995578 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe164df2-18d5-4b40-bead-8cfd5230a783 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:12.996957 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe164df2-18d5-4b40-bead-8cfd5230a783 service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:13.017003 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 728/1455] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:39:15.145411 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb57b2-635c-4a6d-908b-2bdeed0df767 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:15.145942 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb57b2-635c-4a6d-908b-2bdeed0df767 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:15.146072 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb57b2-635c-4a6d-908b-2bdeed0df767 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:15.147462 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 728/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 16:39:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:39:15.153264 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5396d-992a-4e70-9936-c99b1e1b87ef None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:15.153703 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5396d-992a-4e70-9936-c99b1e1b87ef None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:15.153817 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5396d-992a-4e70-9936-c99b1e1b87ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:15.224264 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-2bd5396d-992a-4e70-9936-c99b1e1b87ef None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:15.225512 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-2bd5396d-992a-4e70-9936-c99b1e1b87ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:15.255072 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:15.330929 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 729/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 16:39:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:15.370713 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb57b2-635c-4a6d-908b-2bdeed0df767 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:15.372318 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-b4fb57b2-635c-4a6d-908b-2bdeed0df767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:15.483551 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb57b2-635c-4a6d-908b-2bdeed0df767 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:15.513698 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:15.560940 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ff0058-2ebb-4dc8-87d2-786ed52e1ebc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:15.561602 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ff0058-2ebb-4dc8-87d2-786ed52e1ebc service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:15.561884 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ff0058-2ebb-4dc8-87d2-786ed52e1ebc service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:15.562160 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ff0058-2ebb-4dc8-87d2-786ed52e1ebc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:15.563811 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-92ff0058-2ebb-4dc8-87d2-786ed52e1ebc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:15.673180 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ff0058-2ebb-4dc8-87d2-786ed52e1ebc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:15.674635 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ff0058-2ebb-4dc8-87d2-786ed52e1ebc service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:15.694450 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 729/1458] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:39:17.819932 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841e6e99-fff6-4606-9f33-11e6cadd6cc4 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:17.820557 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841e6e99-fff6-4606-9f33-11e6cadd6cc4 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:17.820557 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841e6e99-fff6-4606-9f33-11e6cadd6cc4 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:17.821877 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 730/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 16:39:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:39:17.827272 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deabe20c-f232-47f1-b387-af2a2df6f3e8 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:17.827774 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deabe20c-f232-47f1-b387-af2a2df6f3e8 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:17.827979 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deabe20c-f232-47f1-b387-af2a2df6f3e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:17.894444 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-deabe20c-f232-47f1-b387-af2a2df6f3e8 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:17.895731 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-deabe20c-f232-47f1-b387-af2a2df6f3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:17.904220 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:18.111278 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 730/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 16:39:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:18.150222 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841e6e99-fff6-4606-9f33-11e6cadd6cc4 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:18.151858 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-841e6e99-fff6-4606-9f33-11e6cadd6cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:18.257637 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841e6e99-fff6-4606-9f33-11e6cadd6cc4 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:18.313875 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:18.333958 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e7d415-cd6f-42b2-9e95-e026bc423865 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:18.334657 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7d415-cd6f-42b2-9e95-e026bc423865 service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:18.334947 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7d415-cd6f-42b2-9e95-e026bc423865 service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:18.335244 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7d415-cd6f-42b2-9e95-e026bc423865 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:18.336925 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-53e7d415-cd6f-42b2-9e95-e026bc423865 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:18.449098 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e7d415-cd6f-42b2-9e95-e026bc423865 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:18.450454 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e7d415-cd6f-42b2-9e95-e026bc423865 service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:18.469230 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 731/1461] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:39:20.596980 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f53e12-e9bb-4a55-aee8-d9552774fa7c None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:20.597559 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f53e12-e9bb-4a55-aee8-d9552774fa7c None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:20.597559 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f53e12-e9bb-4a55-aee8-d9552774fa7c None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:20.599663 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 731/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 16:39:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:39:20.605015 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c29b1-5ae3-4c85-b496-30570647c434 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:20.605444 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c29b1-5ae3-4c85-b496-30570647c434 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:20.605620 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c29b1-5ae3-4c85-b496-30570647c434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:20.676508 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-6b2c29b1-5ae3-4c85-b496-30570647c434 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:20.677898 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-6b2c29b1-5ae3-4c85-b496-30570647c434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:20.717624 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:20.781066 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 732/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 16:39:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:20.820394 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f53e12-e9bb-4a55-aee8-d9552774fa7c None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:20.822625 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-28f53e12-e9bb-4a55-aee8-d9552774fa7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:20.929430 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f53e12-e9bb-4a55-aee8-d9552774fa7c None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:20.966061 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:21.002844 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ec0f32-23e1-45d4-bd81-384485f113e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:21.003535 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ec0f32-23e1-45d4-bd81-384485f113e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:21.003965 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ec0f32-23e1-45d4-bd81-384485f113e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:21.004211 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ec0f32-23e1-45d4-bd81-384485f113e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:21.006728 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-39ec0f32-23e1-45d4-bd81-384485f113e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:21.120715 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ec0f32-23e1-45d4-bd81-384485f113e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:21.122151 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ec0f32-23e1-45d4-bd81-384485f113e2 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:21.140419 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 732/1464] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:39:23.201032 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee3b1b4-b242-4055-9268-a1b136d63211 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:23.201705 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee3b1b4-b242-4055-9268-a1b136d63211 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:23.201705 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee3b1b4-b242-4055-9268-a1b136d63211 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:23.202979 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 733/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 16:39:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:39:23.209095 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec8990c-2ac2-4f7a-8d4b-e101b8968bd6 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:23.209629 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec8990c-2ac2-4f7a-8d4b-e101b8968bd6 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:23.209629 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec8990c-2ac2-4f7a-8d4b-e101b8968bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:23.279328 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-eec8990c-2ac2-4f7a-8d4b-e101b8968bd6 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:23.280579 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-eec8990c-2ac2-4f7a-8d4b-e101b8968bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:23.299029 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:23.378839 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 733/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 16:39:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:23.416569 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee3b1b4-b242-4055-9268-a1b136d63211 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:23.418165 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-fee3b1b4-b242-4055-9268-a1b136d63211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:23.525814 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee3b1b4-b242-4055-9268-a1b136d63211 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:23.597415 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:23.605678 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029f160f-2400-4063-8f7d-491307831022 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:23.606434 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f160f-2400-4063-8f7d-491307831022 service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:23.606862 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f160f-2400-4063-8f7d-491307831022 service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:23.607244 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f160f-2400-4063-8f7d-491307831022 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:23.609399 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-029f160f-2400-4063-8f7d-491307831022 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:23.721813 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029f160f-2400-4063-8f7d-491307831022 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:23.724123 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029f160f-2400-4063-8f7d-491307831022 service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:23.742791 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 734/1467] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:39:25.848744 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4bd944-ccee-4c32-9adb-8e4fe166eedd None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:25.849246 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4bd944-ccee-4c32-9adb-8e4fe166eedd None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:25.849396 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4bd944-ccee-4c32-9adb-8e4fe166eedd None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:25.850778 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 734/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 16:39:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:39:25.856250 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9aac77-3443-42b1-9269-2ef9a37eea5f None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:25.856732 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9aac77-3443-42b1-9269-2ef9a37eea5f None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:25.856839 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9aac77-3443-42b1-9269-2ef9a37eea5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:25.925329 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-3d9aac77-3443-42b1-9269-2ef9a37eea5f None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:25.927325 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-3d9aac77-3443-42b1-9269-2ef9a37eea5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:25.978156 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:26.026674 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 735/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 16:39:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:26.064781 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4bd944-ccee-4c32-9adb-8e4fe166eedd None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:26.066320 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-ea4bd944-ccee-4c32-9adb-8e4fe166eedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:26.171177 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4bd944-ccee-4c32-9adb-8e4fe166eedd None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:26.227040 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:26.255856 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51f5b3e-6b8b-43ec-b467-791ad926029b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:26.256528 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f5b3e-6b8b-43ec-b467-791ad926029b service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:26.256798 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f5b3e-6b8b-43ec-b467-791ad926029b service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:26.257068 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f5b3e-6b8b-43ec-b467-791ad926029b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:26.258724 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a51f5b3e-6b8b-43ec-b467-791ad926029b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:26.369562 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51f5b3e-6b8b-43ec-b467-791ad926029b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:26.370997 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51f5b3e-6b8b-43ec-b467-791ad926029b service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:26.389758 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 735/1470] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:39:36.065787 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=22 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:36.387611 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=66 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:47.232036 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f93326-2f0f-420e-861d-e39541045071 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:47.232611 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f93326-2f0f-420e-861d-e39541045071 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:47.232611 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f93326-2f0f-420e-861d-e39541045071 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:47.234040 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 736/1471] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:39:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:39:47.240924 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb72c40-53c6-432c-b2a4-3c2051c88fe8 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:47.241501 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb72c40-53c6-432c-b2a4-3c2051c88fe8 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:47.241501 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb72c40-53c6-432c-b2a4-3c2051c88fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:47.317278 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-adb72c40-53c6-432c-b2a4-3c2051c88fe8 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:47.318530 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-adb72c40-53c6-432c-b2a4-3c2051c88fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:47.445447 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 736/1472] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:39:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:47.481349 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117846e0-ddd0-4aab-8dd4-1ec2881e4920 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:47.481651 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117846e0-ddd0-4aab-8dd4-1ec2881e4920 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:47.481911 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117846e0-ddd0-4aab-8dd4-1ec2881e4920 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:47.483386 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 737/1473] 10.222.0.26 () {54 vars in 961 bytes} [Mon Jul 18 16:39:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:39:47.489164 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96519c8-8b70-4037-828e-5fc6368cfe71 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:47.489460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96519c8-8b70-4037-828e-5fc6368cfe71 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:47.489885 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96519c8-8b70-4037-828e-5fc6368cfe71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:47.569124 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-b96519c8-8b70-4037-828e-5fc6368cfe71 None None] MFA Rules not processed for user `0887908bcb0d4072bf6acb4cbaf1af83`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:47.570355 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-b96519c8-8b70-4037-828e-5fc6368cfe71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:47.591956 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:47.649451 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 737/1474] 10.222.0.26 () {58 vars in 1058 bytes} [Mon Jul 18 16:39:47 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:47.657086 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117846e0-ddd0-4aab-8dd4-1ec2881e4920 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:47.658661 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-117846e0-ddd0-4aab-8dd4-1ec2881e4920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:47.767032 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117846e0-ddd0-4aab-8dd4-1ec2881e4920 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:47.844572 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f290f6-eba7-4b90-89e4-ff6b9e914553 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0887908bcb0d4072bf6acb4cbaf1af83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:47.845237 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f290f6-eba7-4b90-89e4-ff6b9e914553 service glance] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:47.845517 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f290f6-eba7-4b90-89e4-ff6b9e914553 service glance] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:47.845794 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f290f6-eba7-4b90-89e4-ff6b9e914553 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:47.847443 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-f3f290f6-eba7-4b90-89e4-ff6b9e914553 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:47.904514 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:47.962769 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f290f6-eba7-4b90-89e4-ff6b9e914553 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:47.964261 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f290f6-eba7-4b90-89e4-ff6b9e914553 service glance] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:47.984849 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 738/1475] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:39:50.191123 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9bc98-8840-46c7-83e5-0454ecd2cab3 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:50.191640 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9bc98-8840-46c7-83e5-0454ecd2cab3 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:50.191756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9bc98-8840-46c7-83e5-0454ecd2cab3 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:50.193189 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 738/1476] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:39:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:39:50.198662 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ce8c57-2f71-48f8-967f-4de60fa0aa8a None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:50.199099 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ce8c57-2f71-48f8-967f-4de60fa0aa8a None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:50.199224 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ce8c57-2f71-48f8-967f-4de60fa0aa8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:50.268287 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-72ce8c57-2f71-48f8-967f-4de60fa0aa8a None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:50.270070 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-72ce8c57-2f71-48f8-967f-4de60fa0aa8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:50.373757 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 739/1477] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:39:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:50.397651 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b9bc98-8840-46c7-83e5-0454ecd2cab3 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:50.399215 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f0b9bc98-8840-46c7-83e5-0454ecd2cab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:50.526702 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b9bc98-8840-46c7-83e5-0454ecd2cab3 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:50.584505 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:50.636124 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c322a46-6493-4716-aa5c-84c270d3078a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:50.636789 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c322a46-6493-4716-aa5c-84c270d3078a admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:50.637071 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c322a46-6493-4716-aa5c-84c270d3078a admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:50.637344 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c322a46-6493-4716-aa5c-84c270d3078a admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:50.638108 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c322a46-6493-4716-aa5c-84c270d3078a admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:50.639909 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: WARNING py.warnings [None req-7c322a46-6493-4716-aa5c-84c270d3078a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 16:39:50.639909 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: warnings.warn(msg) Jul 18 16:39:50.639909 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: Jul 18 16:39:50.641046 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c322a46-6493-4716-aa5c-84c270d3078a admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:50.661838 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 739/1478] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 18 16:39:50 2022] GET /identity/v3/projects => generated 2553 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:39:52.776256 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb9194-a8c3-40d4-a590-d4c002024a19 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:52.776855 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb9194-a8c3-40d4-a590-d4c002024a19 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:52.776855 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb9194-a8c3-40d4-a590-d4c002024a19 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:52.778186 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 740/1479] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:39:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:39:52.783591 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871ca0f0-5341-4ae4-b5cc-0a68f08c905d None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:52.784002 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871ca0f0-5341-4ae4-b5cc-0a68f08c905d None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:52.784175 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871ca0f0-5341-4ae4-b5cc-0a68f08c905d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:52.853921 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-871ca0f0-5341-4ae4-b5cc-0a68f08c905d None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:52.855128 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-871ca0f0-5341-4ae4-b5cc-0a68f08c905d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:52.955899 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 740/1480] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:39:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:53.000079 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb9194-a8c3-40d4-a590-d4c002024a19 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:53.001569 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-4abb9194-a8c3-40d4-a590-d4c002024a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:53.012536 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:53.160948 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb9194-a8c3-40d4-a590-d4c002024a19 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:53.243172 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:53.277812 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb3d770-6c2c-4a8d-b4da-7d07d1fcd9c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:53.278477 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb3d770-6c2c-4a8d-b4da-7d07d1fcd9c7 service nova] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:53.278765 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb3d770-6c2c-4a8d-b4da-7d07d1fcd9c7 service nova] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:53.279049 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb3d770-6c2c-4a8d-b4da-7d07d1fcd9c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:53.280758 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-8cb3d770-6c2c-4a8d-b4da-7d07d1fcd9c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:53.374006 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb3d770-6c2c-4a8d-b4da-7d07d1fcd9c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:53.375633 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb3d770-6c2c-4a8d-b4da-7d07d1fcd9c7 service nova] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:53.394011 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 741/1481] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:39:55.633201 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b763a3-130a-4380-aad5-a772edf52cb0 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:55.633726 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b763a3-130a-4380-aad5-a772edf52cb0 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:55.633726 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b763a3-130a-4380-aad5-a772edf52cb0 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:55.635054 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 741/1482] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:39:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:39:55.640385 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ccb97-832b-41fa-b781-f052aa646476 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:55.640852 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ccb97-832b-41fa-b781-f052aa646476 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:55.640964 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ccb97-832b-41fa-b781-f052aa646476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:55.706219 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-c92ccb97-832b-41fa-b781-f052aa646476 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:55.707972 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-c92ccb97-832b-41fa-b781-f052aa646476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:55.758728 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:55.812299 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 742/1483] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:39:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:55.849687 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b763a3-130a-4380-aad5-a772edf52cb0 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:55.851215 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f9b763a3-130a-4380-aad5-a772edf52cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:55.875088 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:55.956233 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b763a3-130a-4380-aad5-a772edf52cb0 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:56.030744 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3625e57-5435-41ff-a86b-f39c7ec11cc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:56.031401 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3625e57-5435-41ff-a86b-f39c7ec11cc2 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:56.031715 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3625e57-5435-41ff-a86b-f39c7ec11cc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:56.031997 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3625e57-5435-41ff-a86b-f39c7ec11cc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:56.033658 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-b3625e57-5435-41ff-a86b-f39c7ec11cc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:56.111374 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:56.146277 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3625e57-5435-41ff-a86b-f39c7ec11cc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:56.147800 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3625e57-5435-41ff-a86b-f39c7ec11cc2 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:56.166878 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 742/1484] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:39:58.266927 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483eb4db-d3b8-4934-8135-46071fc65774 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:58.267457 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483eb4db-d3b8-4934-8135-46071fc65774 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:58.267457 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483eb4db-d3b8-4934-8135-46071fc65774 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:58.268873 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 743/1485] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:39:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:39:58.274920 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac37c7-8809-4f24-a51f-accd69db9857 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:58.275397 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac37c7-8809-4f24-a51f-accd69db9857 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:58.275577 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac37c7-8809-4f24-a51f-accd69db9857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:58.341756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-e1ac37c7-8809-4f24-a51f-accd69db9857 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:39:58.342978 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e1ac37c7-8809-4f24-a51f-accd69db9857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:58.443891 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 743/1486] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:39:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:39:58.483672 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483eb4db-d3b8-4934-8135-46071fc65774 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:39:58.485267 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-483eb4db-d3b8-4934-8135-46071fc65774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:58.593615 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483eb4db-d3b8-4934-8135-46071fc65774 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:39:58.667132 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:39:58.669007 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ebdd12-6852-43a1-a37a-b63ab9e79914 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:39:58.669663 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ebdd12-6852-43a1-a37a-b63ab9e79914 service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:39:58.669950 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ebdd12-6852-43a1-a37a-b63ab9e79914 service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:39:58.670240 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ebdd12-6852-43a1-a37a-b63ab9e79914 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:39:58.672010 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-d3ebdd12-6852-43a1-a37a-b63ab9e79914 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:39:58.782984 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ebdd12-6852-43a1-a37a-b63ab9e79914 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:39:58.784473 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ebdd12-6852-43a1-a37a-b63ab9e79914 service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:39:58.804287 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 744/1487] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:39:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:40:01.026925 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e03588-0597-45ff-90c8-146130b839b8 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:01.027461 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e03588-0597-45ff-90c8-146130b839b8 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:01.027461 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e03588-0597-45ff-90c8-146130b839b8 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:01.028895 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 744/1488] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:40:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:40:01.034287 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d21f71e-bfd5-4c12-a2fa-8f47ab04b22f None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:01.034712 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d21f71e-bfd5-4c12-a2fa-8f47ab04b22f None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:01.034855 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d21f71e-bfd5-4c12-a2fa-8f47ab04b22f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:01.101013 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-4d21f71e-bfd5-4c12-a2fa-8f47ab04b22f None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:40:01.102220 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-4d21f71e-bfd5-4c12-a2fa-8f47ab04b22f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:01.156292 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:01.200926 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 745/1489] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:40:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:40:01.242411 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e03588-0597-45ff-90c8-146130b839b8 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:01.244009 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-89e03588-0597-45ff-90c8-146130b839b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:01.282233 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:01.357932 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e03588-0597-45ff-90c8-146130b839b8 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:01.432994 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5de598c-4b2f-4407-9834-b74d3fcd07e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:01.433643 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de598c-4b2f-4407-9834-b74d3fcd07e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:01.433926 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de598c-4b2f-4407-9834-b74d3fcd07e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:01.434199 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de598c-4b2f-4407-9834-b74d3fcd07e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:01.435870 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d5de598c-4b2f-4407-9834-b74d3fcd07e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:01.522792 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:01.555174 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5de598c-4b2f-4407-9834-b74d3fcd07e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:01.556728 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5de598c-4b2f-4407-9834-b74d3fcd07e8 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:01.578266 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 745/1490] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:40:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:40:04.095935 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065fbcb8-4cfc-43e8-9bdf-f5ad439ba9d2 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:04.096451 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065fbcb8-4cfc-43e8-9bdf-f5ad439ba9d2 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:04.096451 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065fbcb8-4cfc-43e8-9bdf-f5ad439ba9d2 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:04.098368 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 746/1491] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 18 16:40:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:40:04.103801 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98170876-be8d-4012-a013-7bc2d5804dbc None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:04.104304 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98170876-be8d-4012-a013-7bc2d5804dbc None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:04.104399 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98170876-be8d-4012-a013-7bc2d5804dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:04.168833 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-98170876-be8d-4012-a013-7bc2d5804dbc None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:40:04.170042 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-98170876-be8d-4012-a013-7bc2d5804dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:04.268777 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 746/1492] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 18 16:40:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:40:04.309059 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065fbcb8-4cfc-43e8-9bdf-f5ad439ba9d2 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:40:04.310593 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-065fbcb8-4cfc-43e8-9bdf-f5ad439ba9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:04.414968 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065fbcb8-4cfc-43e8-9bdf-f5ad439ba9d2 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:40:04.490145 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf03ff97-6b7d-4435-a419-40e7ecd389c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:40:04.490882 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03ff97-6b7d-4435-a419-40e7ecd389c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:40:04.491168 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03ff97-6b7d-4435-a419-40e7ecd389c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:40:04.491451 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03ff97-6b7d-4435-a419-40e7ecd389c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:40:04.493177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-cf03ff97-6b7d-4435-a419-40e7ecd389c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:40:04.502414 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:04.603824 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf03ff97-6b7d-4435-a419-40e7ecd389c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:40:04.605293 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf03ff97-6b7d-4435-a419-40e7ecd389c1 service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:40:04.624275 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 747/1493] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:40:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:40:14.266897 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=82 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:40:14.621984 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=50 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:47.846617 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4283a8-dcf1-406a-b95f-8295d30e0445 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:47.847100 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4283a8-dcf1-406a-b95f-8295d30e0445 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:47.847208 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4283a8-dcf1-406a-b95f-8295d30e0445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:47.922634 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-7f4283a8-dcf1-406a-b95f-8295d30e0445 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:47.923875 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-7f4283a8-dcf1-406a-b95f-8295d30e0445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:48.023913 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 747/1494] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:41:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:41:48.031988 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5270f949-da08-417e-a2dc-9db6e35cb50a None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:48.032480 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5270f949-da08-417e-a2dc-9db6e35cb50a None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:48.032591 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5270f949-da08-417e-a2dc-9db6e35cb50a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:48.114773 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-5270f949-da08-417e-a2dc-9db6e35cb50a None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:48.116044 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-5270f949-da08-417e-a2dc-9db6e35cb50a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:48.236477 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 748/1495] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:41:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:41:48.242114 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4283a8-dcf1-406a-b95f-8295d30e0445 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:48.243377 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-7f4283a8-dcf1-406a-b95f-8295d30e0445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:48.256132 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4283a8-dcf1-406a-b95f-8295d30e0445 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:48.320784 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:48.356318 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85da16d-3302-4797-a6bf-6b1565ad8024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:48.356895 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85da16d-3302-4797-a6bf-6b1565ad8024 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:48.357175 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85da16d-3302-4797-a6bf-6b1565ad8024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:48.357455 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85da16d-3302-4797-a6bf-6b1565ad8024 admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:48.358221 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85da16d-3302-4797-a6bf-6b1565ad8024 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:41:48.358530 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85da16d-3302-4797-a6bf-6b1565ad8024 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:48.359612 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: WARNING py.warnings [None req-e85da16d-3302-4797-a6bf-6b1565ad8024 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 18 16:41:48.359612 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: warnings.warn(msg) Jul 18 16:41:48.359612 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: Jul 18 16:41:48.360492 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85da16d-3302-4797-a6bf-6b1565ad8024 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:48.372700 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 748/1496] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 16:41:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:41:48.381658 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5270f949-da08-417e-a2dc-9db6e35cb50a None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:48.390837 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5270f949-da08-417e-a2dc-9db6e35cb50a None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:48.456305 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:48.491173 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce2566-5e78-4f39-835e-ac5d9d7e61cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:48.491870 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce2566-5e78-4f39-835e-ac5d9d7e61cd admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:48.492154 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce2566-5e78-4f39-835e-ac5d9d7e61cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:48.492428 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce2566-5e78-4f39-835e-ac5d9d7e61cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:48.493126 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce2566-5e78-4f39-835e-ac5d9d7e61cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:48.494856 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: WARNING py.warnings [None req-01ce2566-5e78-4f39-835e-ac5d9d7e61cd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 16:41:48.494856 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: warnings.warn(msg) Jul 18 16:41:48.494856 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: Jul 18 16:41:48.496207 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce2566-5e78-4f39-835e-ac5d9d7e61cd admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:48.624918 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 749/1497] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:41:48 2022] POST /identity/v3/projects => generated 379 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:41:48.628970 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85da16d-3302-4797-a6bf-6b1565ad8024 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:48.631174 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e85da16d-3302-4797-a6bf-6b1565ad8024 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:48.665158 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85da16d-3302-4797-a6bf-6b1565ad8024 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:48.760151 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0240cf75-5195-42bb-8a61-61f1fa45b89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:48.760840 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240cf75-5195-42bb-8a61-61f1fa45b89c admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:48.761118 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240cf75-5195-42bb-8a61-61f1fa45b89c admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:48.761391 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240cf75-5195-42bb-8a61-61f1fa45b89c admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:48.762082 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0240cf75-5195-42bb-8a61-61f1fa45b89c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:48.763092 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: WARNING py.warnings [None req-0240cf75-5195-42bb-8a61-61f1fa45b89c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 16:41:48.763092 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: warnings.warn(msg) Jul 18 16:41:48.763092 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: Jul 18 16:41:48.764344 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0240cf75-5195-42bb-8a61-61f1fa45b89c admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:48.893528 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 749/1498] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:41:48 2022] POST /identity/v3/users => generated 290 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:41:48.898618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce2566-5e78-4f39-835e-ac5d9d7e61cd admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:48.900357 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-01ce2566-5e78-4f39-835e-ac5d9d7e61cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:48.963701 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce2566-5e78-4f39-835e-ac5d9d7e61cd admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:49.048416 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:49.059700 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5476d43-ffc8-49a3-9fec-8954ca836bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:49.063515 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5476d43-ffc8-49a3-9fec-8954ca836bc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:49.063515 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5476d43-ffc8-49a3-9fec-8954ca836bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:49.063515 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5476d43-ffc8-49a3-9fec-8954ca836bc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:49.063515 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5476d43-ffc8-49a3-9fec-8954ca836bc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:49.063515 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: WARNING py.warnings [None req-a5476d43-ffc8-49a3-9fec-8954ca836bc6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 18 16:41:49.063515 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: warnings.warn(msg) Jul 18 16:41:49.063515 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: Jul 18 16:41:49.064206 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5476d43-ffc8-49a3-9fec-8954ca836bc6 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:49.078683 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 750/1499] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:41:48 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:41:49.084009 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0240cf75-5195-42bb-8a61-61f1fa45b89c admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:49.085647 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-0240cf75-5195-42bb-8a61-61f1fa45b89c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:49.097649 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0240cf75-5195-42bb-8a61-61f1fa45b89c admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:49.125344 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:49.203855 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d83178d-6026-4434-a3dc-a4c6e5816424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:49.204438 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83178d-6026-4434-a3dc-a4c6e5816424 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:49.204721 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83178d-6026-4434-a3dc-a4c6e5816424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:49.204996 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83178d-6026-4434-a3dc-a4c6e5816424 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:49.205723 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d83178d-6026-4434-a3dc-a4c6e5816424 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:49.206754 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: WARNING py.warnings [None req-5d83178d-6026-4434-a3dc-a4c6e5816424 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 18 16:41:49.206754 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: warnings.warn(msg) Jul 18 16:41:49.206754 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: Jul 18 16:41:49.208170 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d83178d-6026-4434-a3dc-a4c6e5816424 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:49.221631 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 750/1500] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:41:49 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:41:49.226485 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5476d43-ffc8-49a3-9fec-8954ca836bc6 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:49.228063 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-a5476d43-ffc8-49a3-9fec-8954ca836bc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:49.265036 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5476d43-ffc8-49a3-9fec-8954ca836bc6 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:49.362824 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6e66fc-16be-4e05-a6da-cb22c08f5bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:49.363781 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6e66fc-16be-4e05-a6da-cb22c08f5bf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:49.364073 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6e66fc-16be-4e05-a6da-cb22c08f5bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:49.364352 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6e66fc-16be-4e05-a6da-cb22c08f5bf7 admin admin] PATH_INFO: `/v3/projects/b873a0dedfe844d095539954ae8f5e6a/users/b204f8adef9f4cd6a79f034c49817c6e/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:49.387902 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6e66fc-16be-4e05-a6da-cb22c08f5bf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b873a0dedfe844d095539954ae8f5e6a, user_id=b204f8adef9f4cd6a79f034c49817c6e, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:49.388954 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: WARNING py.warnings [None req-ee6e66fc-16be-4e05-a6da-cb22c08f5bf7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 16:41:49.388954 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: warnings.warn(msg) Jul 18 16:41:49.388954 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: Jul 18 16:41:49.390904 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6e66fc-16be-4e05-a6da-cb22c08f5bf7 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:49.464048 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 751/1501] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:41:49 2022] PUT /identity/v3/projects/b873a0dedfe844d095539954ae8f5e6a/users/b204f8adef9f4cd6a79f034c49817c6e/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:41:49.476908 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80156c1f-57b2-47d8-8459-60a74d18d73c None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:49.477193 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80156c1f-57b2-47d8-8459-60a74d18d73c None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:49.477465 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80156c1f-57b2-47d8-8459-60a74d18d73c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:49.535452 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-80156c1f-57b2-47d8-8459-60a74d18d73c None None] MFA Rules not processed for user `b204f8adef9f4cd6a79f034c49817c6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:49.537234 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-80156c1f-57b2-47d8-8459-60a74d18d73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:49.619574 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 751/1502] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:41:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:41:49.632982 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d61e7-dcd9-4ab8-9339-585b2c564481 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:49.633274 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d61e7-dcd9-4ab8-9339-585b2c564481 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:49.633552 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d61e7-dcd9-4ab8-9339-585b2c564481 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:49.634932 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 752/1503] 10.222.0.26 () {54 vars in 964 bytes} [Mon Jul 18 16:41:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:41:49.639953 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce596c3e-ad47-400e-a66e-ae420da73bc2 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:49.640270 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce596c3e-ad47-400e-a66e-ae420da73bc2 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:49.640554 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce596c3e-ad47-400e-a66e-ae420da73bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:49.671286 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:49.719039 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-ce596c3e-ad47-400e-a66e-ae420da73bc2 None None] MFA Rules not processed for user `6f2e55df807c43d7951fa4be67ef5e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:49.720229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-ce596c3e-ad47-400e-a66e-ae420da73bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:49.798446 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 752/1504] 10.222.0.26 () {58 vars in 1061 bytes} [Mon Jul 18 16:41:49 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:41:49.806729 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d61e7-dcd9-4ab8-9339-585b2c564481 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:49.808329 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-647d61e7-dcd9-4ab8-9339-585b2c564481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:49.862748 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:49.918881 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d61e7-dcd9-4ab8-9339-585b2c564481 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:49.997005 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad3fe77-03b9-42f1-b128-31204b740746 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:49.997658 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3fe77-03b9-42f1-b128-31204b740746 service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:49.997935 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3fe77-03b9-42f1-b128-31204b740746 service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:49.998269 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3fe77-03b9-42f1-b128-31204b740746 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:49.999951 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-aad3fe77-03b9-42f1-b128-31204b740746 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:50.103111 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad3fe77-03b9-42f1-b128-31204b740746 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:50.104589 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad3fe77-03b9-42f1-b128-31204b740746 service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:50.108176 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:50.124144 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 753/1505] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:41:49 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:41:50.168752 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403be43-c3e8-498a-a8b5-f6c7be338e32 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:50.169237 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403be43-c3e8-498a-a8b5-f6c7be338e32 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:50.169345 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403be43-c3e8-498a-a8b5-f6c7be338e32 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:50.171100 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 753/1506] 10.222.0.26 () {52 vars in 829 bytes} [Mon Jul 18 16:41:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:41:50.174886 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad3fe77-03b9-42f1-b128-31204b740746 service cinder] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:50.176165 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-aad3fe77-03b9-42f1-b128-31204b740746 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:50.298053 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad3fe77-03b9-42f1-b128-31204b740746 service cinder] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:50.403935 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51e31de-e438-4e64-a655-86728b01baf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:50.405147 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e31de-e438-4e64-a655-86728b01baf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:50.405432 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e31de-e438-4e64-a655-86728b01baf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:50.405728 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e31de-e438-4e64-a655-86728b01baf5 admin admin] PATH_INFO: `/v3/users/b204f8adef9f4cd6a79f034c49817c6e` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:50.420563 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51e31de-e438-4e64-a655-86728b01baf5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b204f8adef9f4cd6a79f034c49817c6e)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:50.421851 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: WARNING py.warnings [None req-a51e31de-e438-4e64-a655-86728b01baf5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 16:41:50.421851 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: warnings.warn(msg) Jul 18 16:41:50.421851 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: Jul 18 16:41:50.423208 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51e31de-e438-4e64-a655-86728b01baf5 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:50.435108 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:50.806620 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-a51e31de-e438-4e64-a655-86728b01baf5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b204f8adef9f4cd6a79f034c49817c6e'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:50.959610 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-a51e31de-e438-4e64-a655-86728b01baf5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b204f8adef9f4cd6a79f034c49817c6e'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:50.960084 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-a51e31de-e438-4e64-a655-86728b01baf5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b204f8adef9f4cd6a79f034c49817c6e'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:50.960621 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-a51e31de-e438-4e64-a655-86728b01baf5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b204f8adef9f4cd6a79f034c49817c6e'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:50.974310 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-a51e31de-e438-4e64-a655-86728b01baf5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b204f8adef9f4cd6a79f034c49817c6e'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:51.001515 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 754/1507] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:41:50 2022] DELETE /identity/v3/users/b204f8adef9f4cd6a79f034c49817c6e => generated 0 bytes in 827 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:41:51.006187 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3452fffe-ee0b-42b4-a374-dceb4783929b None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:51.006477 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3452fffe-ee0b-42b4-a374-dceb4783929b None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:51.006762 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3452fffe-ee0b-42b4-a374-dceb4783929b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:51.065397 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-3452fffe-ee0b-42b4-a374-dceb4783929b None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:51.066660 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-3452fffe-ee0b-42b4-a374-dceb4783929b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:51.145320 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:51.157756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 754/1508] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:41:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:41:51.168115 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51e31de-e438-4e64-a655-86728b01baf5 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:51.169326 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-a51e31de-e438-4e64-a655-86728b01baf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:51.276617 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51e31de-e438-4e64-a655-86728b01baf5 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:51.333685 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:51.353451 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c9dc31-864f-402b-8cc1-3939c159252d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:51.354117 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c9dc31-864f-402b-8cc1-3939c159252d service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:51.354404 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c9dc31-864f-402b-8cc1-3939c159252d service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:51.354687 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c9dc31-864f-402b-8cc1-3939c159252d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:51.356395 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-b5c9dc31-864f-402b-8cc1-3939c159252d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:51.482207 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c9dc31-864f-402b-8cc1-3939c159252d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:51.483734 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c9dc31-864f-402b-8cc1-3939c159252d service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:51.502602 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 755/1509] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:41:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:41:51.979461 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3452fffe-ee0b-42b4-a374-dceb4783929b None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:51.980655 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-3452fffe-ee0b-42b4-a374-dceb4783929b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:52.000491 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3452fffe-ee0b-42b4-a374-dceb4783929b None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:52.128773 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fed193-a315-47e1-a55b-217153d65af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:52.131154 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fed193-a315-47e1-a55b-217153d65af5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:52.131627 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fed193-a315-47e1-a55b-217153d65af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:52.132110 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fed193-a315-47e1-a55b-217153d65af5 admin admin] PATH_INFO: `/v3/projects/b873a0dedfe844d095539954ae8f5e6a` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:52.150804 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fed193-a315-47e1-a55b-217153d65af5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b873a0dedfe844d095539954ae8f5e6a)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:52.152224 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: WARNING py.warnings [None req-74fed193-a315-47e1-a55b-217153d65af5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 16:41:52.152224 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: warnings.warn(msg) Jul 18 16:41:52.152224 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: Jul 18 16:41:52.153894 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fed193-a315-47e1-a55b-217153d65af5 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:52.284936 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-74fed193-a315-47e1-a55b-217153d65af5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b873a0dedfe844d095539954ae8f5e6a', 'request_id': 'req-74fed193-a315-47e1-a55b-217153d65af5'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:52.291685 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-74fed193-a315-47e1-a55b-217153d65af5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b873a0dedfe844d095539954ae8f5e6a', 'request_id': 'req-74fed193-a315-47e1-a55b-217153d65af5'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:52.304756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-74fed193-a315-47e1-a55b-217153d65af5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b873a0dedfe844d095539954ae8f5e6a', 'request_id': 'req-74fed193-a315-47e1-a55b-217153d65af5'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:52.388961 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-74fed193-a315-47e1-a55b-217153d65af5 admin admin] The token cache is being invalidate because project b873a0dedfe844d095539954ae8f5e6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:41:52.389249 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-74fed193-a315-47e1-a55b-217153d65af5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:52.389910 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-74fed193-a315-47e1-a55b-217153d65af5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:41:52.391615 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 755/1510] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:41:51 2022] DELETE /identity/v3/projects/b873a0dedfe844d095539954ae8f5e6a => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:42:01.500259 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=62 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:02.386631 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:42.343738 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace745ae-73ff-451c-9561-9aaf131f92f7 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:42.344351 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace745ae-73ff-451c-9561-9aaf131f92f7 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:42.344351 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace745ae-73ff-451c-9561-9aaf131f92f7 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:42.345796 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 756/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 16:43:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:43:42.351843 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e785224-5d74-4ec9-884f-72763574ead5 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:42.352397 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e785224-5d74-4ec9-884f-72763574ead5 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:42.352397 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e785224-5d74-4ec9-884f-72763574ead5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:42.421317 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-4e785224-5d74-4ec9-884f-72763574ead5 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:43:42.422524 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-4e785224-5d74-4ec9-884f-72763574ead5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:42.481312 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 756/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 16:43:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:43:42.638984 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace745ae-73ff-451c-9561-9aaf131f92f7 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:42.641125 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-ace745ae-73ff-451c-9561-9aaf131f92f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:42.764869 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace745ae-73ff-451c-9561-9aaf131f92f7 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:42.770660 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406cdb4d-d88c-4206-92b2-55a435529595 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:42.771930 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406cdb4d-d88c-4206-92b2-55a435529595 service nova] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:42.772207 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406cdb4d-d88c-4206-92b2-55a435529595 service nova] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:42.772489 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406cdb4d-d88c-4206-92b2-55a435529595 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:42.781306 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406cdb4d-d88c-4206-92b2-55a435529595 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:42.782895 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406cdb4d-d88c-4206-92b2-55a435529595 service nova] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:42.792719 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 757/1513] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:43:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:43:43.659569 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73814da-6bc7-437f-aac6-858053c7e321 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:43.660337 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73814da-6bc7-437f-aac6-858053c7e321 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:43.660337 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73814da-6bc7-437f-aac6-858053c7e321 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:43.662177 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 757/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 16:43:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:43:43.670238 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838419a7-5bf7-46ce-b28a-967366200b16 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:43.670746 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838419a7-5bf7-46ce-b28a-967366200b16 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:43.670846 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838419a7-5bf7-46ce-b28a-967366200b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:43.694202 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-838419a7-5bf7-46ce-b28a-967366200b16 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:43:43.695423 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-838419a7-5bf7-46ce-b28a-967366200b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:43.703670 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 758/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 16:43:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:43:43.859951 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73814da-6bc7-437f-aac6-858053c7e321 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:43.877344 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73814da-6bc7-437f-aac6-858053c7e321 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:43.884328 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6a36fb-6204-4454-8aae-dd678507470d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:43.885025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6a36fb-6204-4454-8aae-dd678507470d service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:43.885310 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6a36fb-6204-4454-8aae-dd678507470d service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:43.885587 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6a36fb-6204-4454-8aae-dd678507470d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:43.894693 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6a36fb-6204-4454-8aae-dd678507470d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:43.896968 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6a36fb-6204-4454-8aae-dd678507470d service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:43.908208 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 758/1516] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:43:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:43:44.696239 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b5af8-5bc9-4915-8de9-e44d064e3abe None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:44.696800 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b5af8-5bc9-4915-8de9-e44d064e3abe None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:44.696800 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b5af8-5bc9-4915-8de9-e44d064e3abe None None] PATH_INFO: `/v3` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:44.698157 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 759/1517] 10.222.0.79 () {54 vars in 925 bytes} [Mon Jul 18 16:43:44 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:43:44.705700 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a62be-ce0b-48f9-a548-af7b36c27a83 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:44.706115 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a62be-ce0b-48f9-a548-af7b36c27a83 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:44.706240 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a62be-ce0b-48f9-a548-af7b36c27a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:44.768402 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-ea2a62be-ce0b-48f9-a548-af7b36c27a83 None None] MFA Rules not processed for user `70f704a231a5474e8067a6c2dd88b355`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:43:44.769614 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-ea2a62be-ce0b-48f9-a548-af7b36c27a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:44.848821 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 759/1518] 10.222.0.79 () {58 vars in 1015 bytes} [Mon Jul 18 16:43:44 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:43:49.046658 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbbee73-23da-475d-a149-79042ac65dc0 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:49.047198 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbbee73-23da-475d-a149-79042ac65dc0 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:49.047198 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbbee73-23da-475d-a149-79042ac65dc0 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:49.048617 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 760/1519] 10.222.0.26 () {54 vars in 961 bytes} [Mon Jul 18 16:43:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:43:49.054362 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994048e5-5a89-4431-bbac-ddb93381e653 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:49.054869 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994048e5-5a89-4431-bbac-ddb93381e653 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:49.054977 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994048e5-5a89-4431-bbac-ddb93381e653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:49.064301 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:49.076673 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-994048e5-5a89-4431-bbac-ddb93381e653 None None] MFA Rules not processed for user `70f704a231a5474e8067a6c2dd88b355`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:43:49.077896 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-994048e5-5a89-4431-bbac-ddb93381e653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:49.087652 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 760/1520] 10.222.0.26 () {58 vars in 1058 bytes} [Mon Jul 18 16:43:49 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:43:49.095523 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbbee73-23da-475d-a149-79042ac65dc0 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:49.109888 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbbee73-23da-475d-a149-79042ac65dc0 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:49.115017 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2940062f-152e-437d-935c-28677222e3fc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70f704a231a5474e8067a6c2dd88b355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:49.115721 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2940062f-152e-437d-935c-28677222e3fc service placement] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:49.116007 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2940062f-152e-437d-935c-28677222e3fc service placement] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:49.116288 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2940062f-152e-437d-935c-28677222e3fc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:49.125427 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2940062f-152e-437d-935c-28677222e3fc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:49.127101 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2940062f-152e-437d-935c-28677222e3fc service placement] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:49.137888 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 761/1521] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:43:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:43:55.182908 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d4bbc1-99ee-4d90-92e6-d39b91071f57 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:55.183460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d4bbc1-99ee-4d90-92e6-d39b91071f57 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:55.183460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d4bbc1-99ee-4d90-92e6-d39b91071f57 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:55.184799 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 761/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 16:43:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:43:55.190078 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a26af-01e9-4683-b17c-b65f02a6982e None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:55.190536 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a26af-01e9-4683-b17c-b65f02a6982e None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:55.190661 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a26af-01e9-4683-b17c-b65f02a6982e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:55.211170 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-710a26af-01e9-4683-b17c-b65f02a6982e None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:43:55.212423 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-710a26af-01e9-4683-b17c-b65f02a6982e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:55.220629 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 762/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 16:43:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:43:55.379748 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d4bbc1-99ee-4d90-92e6-d39b91071f57 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:55.395264 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d4bbc1-99ee-4d90-92e6-d39b91071f57 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:55.401892 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e686fc-36e4-4416-a5b2-480f6da3a63f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:55.402533 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e686fc-36e4-4416-a5b2-480f6da3a63f service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:55.402896 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e686fc-36e4-4416-a5b2-480f6da3a63f service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:55.403083 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e686fc-36e4-4416-a5b2-480f6da3a63f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:55.411979 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e686fc-36e4-4416-a5b2-480f6da3a63f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:55.414087 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e686fc-36e4-4416-a5b2-480f6da3a63f service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:55.423637 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 762/1524] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:43:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:43:56.261817 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff772cf-fd0a-41e8-bab5-6b7bf25b2a45 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:56.262433 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff772cf-fd0a-41e8-bab5-6b7bf25b2a45 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:56.262433 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff772cf-fd0a-41e8-bab5-6b7bf25b2a45 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:56.263926 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 763/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 16:43:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:43:56.269230 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6127ea6-8c7f-4b20-9f62-63509dbc97a7 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:56.269747 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6127ea6-8c7f-4b20-9f62-63509dbc97a7 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:56.270008 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6127ea6-8c7f-4b20-9f62-63509dbc97a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:56.289487 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-d6127ea6-8c7f-4b20-9f62-63509dbc97a7 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:43:56.290702 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d6127ea6-8c7f-4b20-9f62-63509dbc97a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:56.298861 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 763/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 16:43:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:43:56.453722 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff772cf-fd0a-41e8-bab5-6b7bf25b2a45 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:56.468561 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff772cf-fd0a-41e8-bab5-6b7bf25b2a45 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:56.474053 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2410c4cc-4282-4239-94d0-2477c809918a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:56.475310 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2410c4cc-4282-4239-94d0-2477c809918a service nova] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:56.475625 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2410c4cc-4282-4239-94d0-2477c809918a service nova] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:56.475942 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2410c4cc-4282-4239-94d0-2477c809918a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:56.484538 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2410c4cc-4282-4239-94d0-2477c809918a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:56.486136 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2410c4cc-4282-4239-94d0-2477c809918a service nova] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:56.498294 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 764/1527] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:43:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:44:00.650427 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d7a6b-bbb4-484f-b854-310a05e7040f None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:44:00.651044 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d7a6b-bbb4-484f-b854-310a05e7040f None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:44:00.651044 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d7a6b-bbb4-484f-b854-310a05e7040f None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:44:00.652395 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 764/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 16:44:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:44:00.657793 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5114f0-1b02-4350-af98-3c24af17fe7a None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:44:00.658240 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5114f0-1b02-4350-af98-3c24af17fe7a None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:44:00.658352 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5114f0-1b02-4350-af98-3c24af17fe7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:44:00.677294 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-3f5114f0-1b02-4350-af98-3c24af17fe7a None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:44:00.678549 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-3f5114f0-1b02-4350-af98-3c24af17fe7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:44:00.686481 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 765/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 16:44:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:44:00.726839 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d7a6b-bbb4-484f-b854-310a05e7040f None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:44:00.728543 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-bd5d7a6b-bbb4-484f-b854-310a05e7040f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:44:00.809941 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d7a6b-bbb4-484f-b854-310a05e7040f None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:44:00.813946 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6254f266-897a-4716-869f-ec5a4350e29f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:44:00.814596 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254f266-897a-4716-869f-ec5a4350e29f service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:44:00.814888 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254f266-897a-4716-869f-ec5a4350e29f service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:44:00.815165 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254f266-897a-4716-869f-ec5a4350e29f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:44:00.826467 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6254f266-897a-4716-869f-ec5a4350e29f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:44:00.827941 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6254f266-897a-4716-869f-ec5a4350e29f service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:44:00.838953 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 765/1530] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:44:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:44:02.909280 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f784d-2a0a-4aef-8552-5df0cc0a1791 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:44:02.909849 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f784d-2a0a-4aef-8552-5df0cc0a1791 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:44:02.909849 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f784d-2a0a-4aef-8552-5df0cc0a1791 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:44:02.911284 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 766/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 16:44:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:44:02.917538 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b079b-d2a3-4f64-b085-dc49fb01dbe9 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:44:02.918030 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b079b-d2a3-4f64-b085-dc49fb01dbe9 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:44:02.918141 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b079b-d2a3-4f64-b085-dc49fb01dbe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:44:02.936358 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-4a7b079b-d2a3-4f64-b085-dc49fb01dbe9 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:44:02.937852 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-4a7b079b-d2a3-4f64-b085-dc49fb01dbe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:44:02.946403 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 766/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 16:44:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:44:02.983350 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f784d-2a0a-4aef-8552-5df0cc0a1791 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:44:02.994314 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f784d-2a0a-4aef-8552-5df0cc0a1791 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:44:02.997865 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7cf514-efe3-4873-b18e-fc049deed9a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:44:02.998554 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7cf514-efe3-4873-b18e-fc049deed9a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:44:02.998838 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7cf514-efe3-4873-b18e-fc049deed9a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:44:02.999114 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7cf514-efe3-4873-b18e-fc049deed9a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:44:03.009114 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7cf514-efe3-4873-b18e-fc049deed9a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:44:03.010553 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7cf514-efe3-4873-b18e-fc049deed9a7 service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:44:03.020309 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 767/1533] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:44:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:44:06.655541 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86733ec-c3fd-4457-9eed-d34199ca6e2d None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:44:06.656156 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86733ec-c3fd-4457-9eed-d34199ca6e2d None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:44:06.656156 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86733ec-c3fd-4457-9eed-d34199ca6e2d None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:44:06.657477 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 767/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 16:44:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:44:06.663606 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe719d-8d06-4690-93a9-0092da08addb None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:44:06.664149 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe719d-8d06-4690-93a9-0092da08addb None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:44:06.664149 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe719d-8d06-4690-93a9-0092da08addb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:44:06.682693 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-8ffe719d-8d06-4690-93a9-0092da08addb None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:44:06.683930 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-8ffe719d-8d06-4690-93a9-0092da08addb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:44:06.692100 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 768/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 16:44:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:44:07.010321 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86733ec-c3fd-4457-9eed-d34199ca6e2d None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:44:07.012064 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-c86733ec-c3fd-4457-9eed-d34199ca6e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:44:07.093155 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86733ec-c3fd-4457-9eed-d34199ca6e2d None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:44:07.096943 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc99ebd-63d3-432a-bc3e-04c01ffa03fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:44:07.097589 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc99ebd-63d3-432a-bc3e-04c01ffa03fd service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:44:07.097867 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc99ebd-63d3-432a-bc3e-04c01ffa03fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:44:07.098164 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc99ebd-63d3-432a-bc3e-04c01ffa03fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:44:07.108911 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc99ebd-63d3-432a-bc3e-04c01ffa03fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:44:07.110349 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc99ebd-63d3-432a-bc3e-04c01ffa03fd service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:44:07.120552 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 768/1536] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:44:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:44:09.329696 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7bf6d3-2665-450f-8a07-a61e60f9f6b6 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:44:09.330282 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7bf6d3-2665-450f-8a07-a61e60f9f6b6 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:44:09.330282 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7bf6d3-2665-450f-8a07-a61e60f9f6b6 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:44:09.331682 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 769/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 16:44:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:44:09.338124 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c313753-274f-4560-88d2-38e4ee00c516 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:44:09.338626 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c313753-274f-4560-88d2-38e4ee00c516 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:44:09.338813 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c313753-274f-4560-88d2-38e4ee00c516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:44:09.358198 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-5c313753-274f-4560-88d2-38e4ee00c516 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:44:09.359423 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-5c313753-274f-4560-88d2-38e4ee00c516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:44:09.367805 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 769/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 16:44:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:44:09.678059 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7bf6d3-2665-450f-8a07-a61e60f9f6b6 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:44:09.679865 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-bb7bf6d3-2665-450f-8a07-a61e60f9f6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:44:09.696476 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7bf6d3-2665-450f-8a07-a61e60f9f6b6 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:44:09.700915 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c35881-cf2f-4a38-9615-44c4e30aec7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:44:09.701766 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c35881-cf2f-4a38-9615-44c4e30aec7e service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:44:09.702376 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c35881-cf2f-4a38-9615-44c4e30aec7e service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:44:09.702571 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c35881-cf2f-4a38-9615-44c4e30aec7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:44:09.714370 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c35881-cf2f-4a38-9615-44c4e30aec7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:44:09.715876 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c35881-cf2f-4a38-9615-44c4e30aec7e service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:44:09.726624 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 770/1539] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:44:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:44:19.352958 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=87 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:44:19.721749 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=89 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:45:40.197072 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e3137-67a1-4159-b399-90917db22d49 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:40.197791 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e3137-67a1-4159-b399-90917db22d49 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:40.197791 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e3137-67a1-4159-b399-90917db22d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:40.228698 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-1e4e3137-67a1-4159-b399-90917db22d49 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:40.230037 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-1e4e3137-67a1-4159-b399-90917db22d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:40.245836 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 770/1540] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:45:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:45:40.255017 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5984e5aa-f4b4-40f1-8521-5faaaa41a358 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:40.255760 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5984e5aa-f4b4-40f1-8521-5faaaa41a358 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:40.255760 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5984e5aa-f4b4-40f1-8521-5faaaa41a358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:40.289366 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-5984e5aa-f4b4-40f1-8521-5faaaa41a358 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:40.290624 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-5984e5aa-f4b4-40f1-8521-5faaaa41a358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:40.302677 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 771/1541] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:45:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:45:40.307423 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4e3137-67a1-4159-b399-90917db22d49 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:40.322671 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4e3137-67a1-4159-b399-90917db22d49 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:40.336578 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c99247-704f-4ab6-840f-a5d5cfac3e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:40.337331 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c99247-704f-4ab6-840f-a5d5cfac3e3b admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:40.337620 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c99247-704f-4ab6-840f-a5d5cfac3e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:40.337898 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c99247-704f-4ab6-840f-a5d5cfac3e3b admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:40.338646 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c99247-704f-4ab6-840f-a5d5cfac3e3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:45:40.338957 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c99247-704f-4ab6-840f-a5d5cfac3e3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:40.340635 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c99247-704f-4ab6-840f-a5d5cfac3e3b admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:40.365955 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 771/1542] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 16:45:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:45:40.374400 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5984e5aa-f4b4-40f1-8521-5faaaa41a358 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:40.388931 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5984e5aa-f4b4-40f1-8521-5faaaa41a358 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:40.394209 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f02f7-2f20-44ce-ae81-c2751b6c96c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:40.394990 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f02f7-2f20-44ce-ae81-c2751b6c96c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:40.395330 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f02f7-2f20-44ce-ae81-c2751b6c96c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:40.399815 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f02f7-2f20-44ce-ae81-c2751b6c96c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:40.400693 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872f02f7-2f20-44ce-ae81-c2751b6c96c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:40.402767 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872f02f7-2f20-44ce-ae81-c2751b6c96c3 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:40.571539 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 772/1543] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:45:40 2022] POST /identity/v3/projects => generated 379 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:45:40.575839 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c99247-704f-4ab6-840f-a5d5cfac3e3b admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:40.589678 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c99247-704f-4ab6-840f-a5d5cfac3e3b admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:40.664925 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1013f068-f12a-4696-85a4-868850d38583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:40.665625 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013f068-f12a-4696-85a4-868850d38583 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:40.665914 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013f068-f12a-4696-85a4-868850d38583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:40.666193 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013f068-f12a-4696-85a4-868850d38583 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:40.666969 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1013f068-f12a-4696-85a4-868850d38583 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:40.669054 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1013f068-f12a-4696-85a4-868850d38583 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:40.779189 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 772/1544] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:45:40 2022] POST /identity/v3/users => generated 290 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:45:40.783263 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f02f7-2f20-44ce-ae81-c2751b6c96c3 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:40.795286 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f02f7-2f20-44ce-ae81-c2751b6c96c3 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:40.805389 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791bbaa1-62fc-4cb3-b756-0892d26da094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:40.806544 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791bbaa1-62fc-4cb3-b756-0892d26da094 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:40.806840 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791bbaa1-62fc-4cb3-b756-0892d26da094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:40.807191 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791bbaa1-62fc-4cb3-b756-0892d26da094 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:40.808097 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791bbaa1-62fc-4cb3-b756-0892d26da094 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:40.810844 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791bbaa1-62fc-4cb3-b756-0892d26da094 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:40.832144 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 773/1545] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:45:40 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:45:40.839471 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1013f068-f12a-4696-85a4-868850d38583 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:40.855080 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1013f068-f12a-4696-85a4-868850d38583 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:40.862306 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d88e20b-18b1-43a8-a109-9fb0b922834f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:40.863254 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88e20b-18b1-43a8-a109-9fb0b922834f admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:40.863638 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88e20b-18b1-43a8-a109-9fb0b922834f admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:40.864083 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88e20b-18b1-43a8-a109-9fb0b922834f admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:40.864933 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d88e20b-18b1-43a8-a109-9fb0b922834f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:40.867058 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d88e20b-18b1-43a8-a109-9fb0b922834f admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:40.882494 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 773/1546] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:45:40 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:45:40.887253 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791bbaa1-62fc-4cb3-b756-0892d26da094 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:40.907333 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791bbaa1-62fc-4cb3-b756-0892d26da094 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:40.918524 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b15492d-8db1-435f-89d1-8806f78279f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:40.920318 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b15492d-8db1-435f-89d1-8806f78279f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:40.921017 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b15492d-8db1-435f-89d1-8806f78279f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:40.921682 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b15492d-8db1-435f-89d1-8806f78279f7 admin admin] PATH_INFO: `/v3/projects/540cf87ca5364bf598ab16cdc3903a1c/users/87094997b3b94ebea3de60446795fb7a/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:40.958092 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b15492d-8db1-435f-89d1-8806f78279f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=540cf87ca5364bf598ab16cdc3903a1c, user_id=87094997b3b94ebea3de60446795fb7a, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:40.962482 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b15492d-8db1-435f-89d1-8806f78279f7 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:41.050168 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 774/1547] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:45:40 2022] PUT /identity/v3/projects/540cf87ca5364bf598ab16cdc3903a1c/users/87094997b3b94ebea3de60446795fb7a/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:45:41.055478 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b8b36f-6521-4c94-b3b2-b1a7a8894c9b None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:41.059587 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b8b36f-6521-4c94-b3b2-b1a7a8894c9b None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:41.059587 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b8b36f-6521-4c94-b3b2-b1a7a8894c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:41.080460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-93b8b36f-6521-4c94-b3b2-b1a7a8894c9b None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:41.081791 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-93b8b36f-6521-4c94-b3b2-b1a7a8894c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:41.092354 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c4e927-f199-44ee-b69a-dd2f11accb9b None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:41.093041 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c4e927-f199-44ee-b69a-dd2f11accb9b None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:41.093550 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c4e927-f199-44ee-b69a-dd2f11accb9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:41.127472 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-48c4e927-f199-44ee-b69a-dd2f11accb9b None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:41.129185 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-48c4e927-f199-44ee-b69a-dd2f11accb9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:41.201760 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 774/1548] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:45:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:45:41.203400 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88b95a1-60db-46d3-8208-14aba22458e1 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:41.203759 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88b95a1-60db-46d3-8208-14aba22458e1 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:41.204041 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88b95a1-60db-46d3-8208-14aba22458e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:41.245205 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-e88b95a1-60db-46d3-8208-14aba22458e1 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:41.246738 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e88b95a1-60db-46d3-8208-14aba22458e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:41.257753 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 775/1549] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:45:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:45:41.259872 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6880f-216e-4b67-9ede-6f7ee43e36c5 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:41.260349 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6880f-216e-4b67-9ede-6f7ee43e36c5 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:41.261019 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6880f-216e-4b67-9ede-6f7ee43e36c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:41.283556 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 775/1550] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:45:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:45:41.283556 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c4e927-f199-44ee-b69a-dd2f11accb9b None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:41.290790 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-d4a6880f-216e-4b67-9ede-6f7ee43e36c5 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:41.292828 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d4a6880f-216e-4b67-9ede-6f7ee43e36c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:41.301590 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 776/1551] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:45:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:45:41.302839 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f435c47d-95b7-430e-9d9e-04301c88d8ce None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:41.303143 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f435c47d-95b7-430e-9d9e-04301c88d8ce None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:41.304926 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f435c47d-95b7-430e-9d9e-04301c88d8ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:41.327329 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-f435c47d-95b7-430e-9d9e-04301c88d8ce None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:41.328584 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f435c47d-95b7-430e-9d9e-04301c88d8ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:41.338700 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 777/1552] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:45:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:45:41.340624 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a1584-d82d-4e50-bfca-9919bcaee7d0 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:41.341020 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a1584-d82d-4e50-bfca-9919bcaee7d0 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:41.341414 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a1584-d82d-4e50-bfca-9919bcaee7d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:41.361919 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-3b6a1584-d82d-4e50-bfca-9919bcaee7d0 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:41.362813 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-3b6a1584-d82d-4e50-bfca-9919bcaee7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:41.371801 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 778/1553] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:45:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:45:41.373072 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ec07d-43c1-4116-8263-430fb9dc5996 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:41.373362 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ec07d-43c1-4116-8263-430fb9dc5996 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:41.373645 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ec07d-43c1-4116-8263-430fb9dc5996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:41.387923 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c4e927-f199-44ee-b69a-dd2f11accb9b None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:41.391718 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60792698-e23e-4f55-8d3b-dd047d569b39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:41.392391 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60792698-e23e-4f55-8d3b-dd047d569b39 service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:41.392673 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60792698-e23e-4f55-8d3b-dd047d569b39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:41.393021 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60792698-e23e-4f55-8d3b-dd047d569b39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:41.403349 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60792698-e23e-4f55-8d3b-dd047d569b39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:41.404979 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60792698-e23e-4f55-8d3b-dd047d569b39 service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:41.405558 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-b98ec07d-43c1-4116-8263-430fb9dc5996 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:41.406780 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-b98ec07d-43c1-4116-8263-430fb9dc5996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:41.421396 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 779/1554] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:45:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:45:41.423473 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 776/1555] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:45:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:45:41.423859 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87920375-c17b-4e22-81bb-0b3ee4c3388a None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:41.424315 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87920375-c17b-4e22-81bb-0b3ee4c3388a None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:41.424728 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87920375-c17b-4e22-81bb-0b3ee4c3388a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:41.425130 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36fbc97-ae5a-4b33-b0ee-6b0762affbab None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:41.425429 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36fbc97-ae5a-4b33-b0ee-6b0762affbab None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:41.425864 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36fbc97-ae5a-4b33-b0ee-6b0762affbab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:41.447314 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-f36fbc97-ae5a-4b33-b0ee-6b0762affbab None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:41.463656 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-f36fbc97-ae5a-4b33-b0ee-6b0762affbab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:41.464710 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-87920375-c17b-4e22-81bb-0b3ee4c3388a None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:41.464710 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-87920375-c17b-4e22-81bb-0b3ee4c3388a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:41.464898 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 777/1556] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:45:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:45:41.464898 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97988ce9-9315-4f79-bc72-a8813e7f0b11 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:41.464898 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97988ce9-9315-4f79-bc72-a8813e7f0b11 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:41.464898 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97988ce9-9315-4f79-bc72-a8813e7f0b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:41.468309 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 780/1557] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:45:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:45:41.469757 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f142534a-1cef-4441-8db7-face0c23852e None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:41.470135 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f142534a-1cef-4441-8db7-face0c23852e None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:41.470418 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f142534a-1cef-4441-8db7-face0c23852e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:41.489177 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-f142534a-1cef-4441-8db7-face0c23852e None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:41.490081 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-97988ce9-9315-4f79-bc72-a8813e7f0b11 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:41.490402 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f142534a-1cef-4441-8db7-face0c23852e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:41.491266 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-97988ce9-9315-4f79-bc72-a8813e7f0b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:41.499539 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 781/1558] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:45:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:45:41.500606 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd58bb-8e6d-493d-9784-28feac0cf626 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:41.500897 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd58bb-8e6d-493d-9784-28feac0cf626 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:41.501174 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd58bb-8e6d-493d-9784-28feac0cf626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:41.509341 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 778/1559] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:45:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:45:41.511276 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db05c2c3-2b64-44df-b76f-2ac3bf4365b7 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:41.511772 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db05c2c3-2b64-44df-b76f-2ac3bf4365b7 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:41.512276 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db05c2c3-2b64-44df-b76f-2ac3bf4365b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:41.521046 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-12cd58bb-8e6d-493d-9784-28feac0cf626 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:41.522286 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-12cd58bb-8e6d-493d-9784-28feac0cf626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:41.536870 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 782/1560] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:45:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:45:41.537285 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-db05c2c3-2b64-44df-b76f-2ac3bf4365b7 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:41.537460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cd58bb-8e6d-493d-9784-28feac0cf626 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:41.537632 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-db05c2c3-2b64-44df-b76f-2ac3bf4365b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:41.548724 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cd58bb-8e6d-493d-9784-28feac0cf626 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:41.551052 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 779/1561] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:45:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:45:41.552600 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db05c2c3-2b64-44df-b76f-2ac3bf4365b7 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:41.559540 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602d87a1-af9d-49f4-8f76-dae5f594b83b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:41.559540 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d87a1-af9d-49f4-8f76-dae5f594b83b admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:41.559540 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d87a1-af9d-49f4-8f76-dae5f594b83b admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:41.559540 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d87a1-af9d-49f4-8f76-dae5f594b83b admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:41.559905 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602d87a1-af9d-49f4-8f76-dae5f594b83b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:45:41.559905 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602d87a1-af9d-49f4-8f76-dae5f594b83b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:41.560922 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602d87a1-af9d-49f4-8f76-dae5f594b83b admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:41.562943 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:45:41.575234 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 783/1562] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 16:45:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:45:41.576665 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db05c2c3-2b64-44df-b76f-2ac3bf4365b7 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:41.576809 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8f6025-1923-4b27-888a-c6b247aed813 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:41.577019 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8f6025-1923-4b27-888a-c6b247aed813 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:41.577933 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8f6025-1923-4b27-888a-c6b247aed813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:41.584883 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3376b104-8276-4588-b39c-0c3b511870ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:41.587675 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376b104-8276-4588-b39c-0c3b511870ab admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:41.588110 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376b104-8276-4588-b39c-0c3b511870ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:41.588556 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376b104-8276-4588-b39c-0c3b511870ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:41.589491 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3376b104-8276-4588-b39c-0c3b511870ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:45:41.590324 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3376b104-8276-4588-b39c-0c3b511870ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:41.591369 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: WARNING py.warnings [None req-3376b104-8276-4588-b39c-0c3b511870ab admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 18 16:45:41.591369 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: warnings.warn(msg) Jul 18 16:45:41.591369 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: Jul 18 16:45:41.592938 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3376b104-8276-4588-b39c-0c3b511870ab admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:41.598569 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-8f8f6025-1923-4b27-888a-c6b247aed813 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:41.599817 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-8f8f6025-1923-4b27-888a-c6b247aed813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:41.603691 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:45:41.609889 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 784/1563] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:45:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:45:41.611166 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6424bfcf-0e75-43f0-91be-2970c36fca1d None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:41.611628 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6424bfcf-0e75-43f0-91be-2970c36fca1d None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:41.612004 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6424bfcf-0e75-43f0-91be-2970c36fca1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:41.619753 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 780/1564] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 16:45:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:45:41.620657 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3376b104-8276-4588-b39c-0c3b511870ab admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:41.631388 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3376b104-8276-4588-b39c-0c3b511870ab admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:41.637642 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca97b4e-72e7-451c-a923-13689b7481c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:41.638389 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca97b4e-72e7-451c-a923-13689b7481c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:41.638670 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca97b4e-72e7-451c-a923-13689b7481c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:41.639001 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca97b4e-72e7-451c-a923-13689b7481c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:41.639794 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca97b4e-72e7-451c-a923-13689b7481c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:45:41.640155 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca97b4e-72e7-451c-a923-13689b7481c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:41.640661 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-6424bfcf-0e75-43f0-91be-2970c36fca1d None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:41.641809 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca97b4e-72e7-451c-a923-13689b7481c1 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:41.641994 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-6424bfcf-0e75-43f0-91be-2970c36fca1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:41.654804 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 785/1565] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:45:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:45:41.656586 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6424bfcf-0e75-43f0-91be-2970c36fca1d None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:41.666236 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 781/1566] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 16:45:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:45:41.667396 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca97b4e-72e7-451c-a923-13689b7481c1 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:41.674126 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6424bfcf-0e75-43f0-91be-2970c36fca1d None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:41.679760 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61744ca4-865a-4f2d-ac62-7c4d3e0befaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:41.681370 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61744ca4-865a-4f2d-ac62-7c4d3e0befaa admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:41.681370 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61744ca4-865a-4f2d-ac62-7c4d3e0befaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:41.681370 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61744ca4-865a-4f2d-ac62-7c4d3e0befaa admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:41.682119 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61744ca4-865a-4f2d-ac62-7c4d3e0befaa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:45:41.682269 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca97b4e-72e7-451c-a923-13689b7481c1 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:41.682428 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61744ca4-865a-4f2d-ac62-7c4d3e0befaa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:41.684080 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61744ca4-865a-4f2d-ac62-7c4d3e0befaa admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:41.688633 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a1cb8a-b68f-4272-b6d3-c7f3458754fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:41.689558 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a1cb8a-b68f-4272-b6d3-c7f3458754fa admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:41.689949 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a1cb8a-b68f-4272-b6d3-c7f3458754fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:41.690334 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a1cb8a-b68f-4272-b6d3-c7f3458754fa admin admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:41.691372 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a1cb8a-b68f-4272-b6d3-c7f3458754fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:45:41.691842 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a1cb8a-b68f-4272-b6d3-c7f3458754fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:41.694443 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a1cb8a-b68f-4272-b6d3-c7f3458754fa admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:41.699359 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 786/1567] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 16:45:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:45:41.700261 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61744ca4-865a-4f2d-ac62-7c4d3e0befaa admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:41.713297 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 782/1568] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 16:45:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:45:41.713297 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a1cb8a-b68f-4272-b6d3-c7f3458754fa admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:41.716128 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61744ca4-865a-4f2d-ac62-7c4d3e0befaa admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:41.725911 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d0f850-c9e9-483d-b164-b82f12a38cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:41.726752 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0f850-c9e9-483d-b164-b82f12a38cb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:41.727143 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0f850-c9e9-483d-b164-b82f12a38cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:41.727265 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a1cb8a-b68f-4272-b6d3-c7f3458754fa admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:41.727445 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0f850-c9e9-483d-b164-b82f12a38cb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:41.728219 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d0f850-c9e9-483d-b164-b82f12a38cb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:41.729283 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: WARNING py.warnings [None req-b7d0f850-c9e9-483d-b164-b82f12a38cb6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 16:45:41.729283 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: warnings.warn(msg) Jul 18 16:45:41.729283 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: Jul 18 16:45:41.730486 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d0f850-c9e9-483d-b164-b82f12a38cb6 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:41.735409 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a23732-f383-4bfe-a7b4-f2458047d9de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:41.736119 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a23732-f383-4bfe-a7b4-f2458047d9de admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:41.736420 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a23732-f383-4bfe-a7b4-f2458047d9de admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:41.736693 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a23732-f383-4bfe-a7b4-f2458047d9de admin admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:41.737441 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a23732-f383-4bfe-a7b4-f2458047d9de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:45:41.737743 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a23732-f383-4bfe-a7b4-f2458047d9de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:41.739345 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a23732-f383-4bfe-a7b4-f2458047d9de admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:41.754854 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 783/1569] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 16:45:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:45:41.755708 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a23732-f383-4bfe-a7b4-f2458047d9de admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:41.766078 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a23732-f383-4bfe-a7b4-f2458047d9de admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:41.770719 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25585899-cb55-4ff4-a471-e740a5692e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:41.771405 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25585899-cb55-4ff4-a471-e740a5692e23 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:41.771711 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25585899-cb55-4ff4-a471-e740a5692e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:41.771985 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25585899-cb55-4ff4-a471-e740a5692e23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:41.772723 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25585899-cb55-4ff4-a471-e740a5692e23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:41.775169 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25585899-cb55-4ff4-a471-e740a5692e23 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:41.841126 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 787/1570] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:45:41 2022] POST /identity/v3/projects => generated 355 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:45:41.842097 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d0f850-c9e9-483d-b164-b82f12a38cb6 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:41.903243 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 784/1571] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:45:41 2022] POST /identity/v3/projects => generated 385 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:45:41.904623 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25585899-cb55-4ff4-a471-e740a5692e23 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:41.917606 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d0f850-c9e9-483d-b164-b82f12a38cb6 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:41.982739 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25585899-cb55-4ff4-a471-e740a5692e23 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:41.983344 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9964035-9a8a-4572-9f55-981b12d256ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:41.984057 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9964035-9a8a-4572-9f55-981b12d256ce admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:41.984342 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9964035-9a8a-4572-9f55-981b12d256ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:41.984621 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9964035-9a8a-4572-9f55-981b12d256ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:41.985370 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9964035-9a8a-4572-9f55-981b12d256ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:45:41.985679 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9964035-9a8a-4572-9f55-981b12d256ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:41.987322 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9964035-9a8a-4572-9f55-981b12d256ce admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:41.988939 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aaba45-7b57-420c-851c-7f3a28f9e706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:41.992999 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aaba45-7b57-420c-851c-7f3a28f9e706 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:41.992999 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aaba45-7b57-420c-851c-7f3a28f9e706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:41.992999 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aaba45-7b57-420c-851c-7f3a28f9e706 admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:41.992999 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aaba45-7b57-420c-851c-7f3a28f9e706 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:41.994037 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aaba45-7b57-420c-851c-7f3a28f9e706 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:42.000091 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 788/1572] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 16:45:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:45:42.008343 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9964035-9a8a-4572-9f55-981b12d256ce admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:42.019005 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9964035-9a8a-4572-9f55-981b12d256ce admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:42.027569 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda6747c-4cf5-4793-b8d2-0690bb360b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:42.027569 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda6747c-4cf5-4793-b8d2-0690bb360b23 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:42.027569 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda6747c-4cf5-4793-b8d2-0690bb360b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:42.027569 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda6747c-4cf5-4793-b8d2-0690bb360b23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:42.028278 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda6747c-4cf5-4793-b8d2-0690bb360b23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:42.029234 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda6747c-4cf5-4793-b8d2-0690bb360b23 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:42.113108 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 785/1573] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:45:41 2022] POST /identity/v3/projects => generated 375 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:45:42.113947 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aaba45-7b57-420c-851c-7f3a28f9e706 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:42.190183 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 789/1574] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:45:42 2022] POST /identity/v3/projects => generated 403 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:45:42.191039 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda6747c-4cf5-4793-b8d2-0690bb360b23 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:42.200623 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aaba45-7b57-420c-851c-7f3a28f9e706 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:42.228994 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:45:42.269755 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16aea01-42af-494a-b947-5b2ecbe820f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:42.270429 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16aea01-42af-494a-b947-5b2ecbe820f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:42.270714 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16aea01-42af-494a-b947-5b2ecbe820f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:42.270991 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16aea01-42af-494a-b947-5b2ecbe820f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:42.271728 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16aea01-42af-494a-b947-5b2ecbe820f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:42.274908 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16aea01-42af-494a-b947-5b2ecbe820f4 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:42.279407 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda6747c-4cf5-4793-b8d2-0690bb360b23 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:42.286093 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa44eb2-a448-4ec7-ac0c-8b02f81e9663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:42.286927 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa44eb2-a448-4ec7-ac0c-8b02f81e9663 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:42.287218 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa44eb2-a448-4ec7-ac0c-8b02f81e9663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:42.287516 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa44eb2-a448-4ec7-ac0c-8b02f81e9663 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:42.288268 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa44eb2-a448-4ec7-ac0c-8b02f81e9663 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:42.291026 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa44eb2-a448-4ec7-ac0c-8b02f81e9663 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:42.304378 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:45:42.404308 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 786/1575] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:45:42 2022] POST /identity/v3/projects => generated 405 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:45:42.405574 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16aea01-42af-494a-b947-5b2ecbe820f4 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:42.434446 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 790/1576] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:45:42 2022] POST /identity/v3/projects => generated 393 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:45:42.435631 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa44eb2-a448-4ec7-ac0c-8b02f81e9663 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:42.481592 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16aea01-42af-494a-b947-5b2ecbe820f4 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:42.513302 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa44eb2-a448-4ec7-ac0c-8b02f81e9663 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:42.520194 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23ca1f3-7fdb-445f-8ff0-551cac6ded26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:42.521143 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ca1f3-7fdb-445f-8ff0-551cac6ded26 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:42.521624 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ca1f3-7fdb-445f-8ff0-551cac6ded26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:42.522088 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ca1f3-7fdb-445f-8ff0-551cac6ded26 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:42.523306 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23ca1f3-7fdb-445f-8ff0-551cac6ded26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:42.526715 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23ca1f3-7fdb-445f-8ff0-551cac6ded26 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:42.545161 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd6e302-fa6c-43fc-9188-09dc688aaa65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:42.549849 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd6e302-fa6c-43fc-9188-09dc688aaa65 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:42.549849 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd6e302-fa6c-43fc-9188-09dc688aaa65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:42.549849 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd6e302-fa6c-43fc-9188-09dc688aaa65 admin admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:42.549849 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd6e302-fa6c-43fc-9188-09dc688aaa65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:42.549849 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: WARNING py.warnings [None req-ecd6e302-fa6c-43fc-9188-09dc688aaa65 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 16:45:42.549849 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: warnings.warn(msg) Jul 18 16:45:42.549849 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: Jul 18 16:45:42.550530 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd6e302-fa6c-43fc-9188-09dc688aaa65 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:42.667629 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 787/1577] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:45:42 2022] POST /identity/v3/users => generated 278 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:45:42.667629 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd6e302-fa6c-43fc-9188-09dc688aaa65 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:42.667948 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 791/1578] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:45:42 2022] POST /identity/v3/users => generated 293 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:45:42.667948 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23ca1f3-7fdb-445f-8ff0-551cac6ded26 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:42.678452 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd6e302-fa6c-43fc-9188-09dc688aaa65 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:42.683675 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23ca1f3-7fdb-445f-8ff0-551cac6ded26 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:42.684137 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffcbabd-a930-4b1d-976f-197de3f009b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:42.685033 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffcbabd-a930-4b1d-976f-197de3f009b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:42.685256 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffcbabd-a930-4b1d-976f-197de3f009b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:42.685783 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffcbabd-a930-4b1d-976f-197de3f009b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:42.686748 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffcbabd-a930-4b1d-976f-197de3f009b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:42.688824 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffcbabd-a930-4b1d-976f-197de3f009b8 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:42.689010 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cff06c-f85c-4bdd-9749-0691bebcfd77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:42.690173 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cff06c-f85c-4bdd-9749-0691bebcfd77 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:42.690633 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cff06c-f85c-4bdd-9749-0691bebcfd77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:42.690986 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cff06c-f85c-4bdd-9749-0691bebcfd77 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:42.691815 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cff06c-f85c-4bdd-9749-0691bebcfd77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:42.694100 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cff06c-f85c-4bdd-9749-0691bebcfd77 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:42.798139 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 792/1579] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:45:42 2022] POST /identity/v3/users => generated 288 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:45:42.801918 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 788/1580] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:45:42 2022] POST /identity/v3/projects => generated 375 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:45:42.802085 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cff06c-f85c-4bdd-9749-0691bebcfd77 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:42.802199 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffcbabd-a930-4b1d-976f-197de3f009b8 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:42.817528 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cff06c-f85c-4bdd-9749-0691bebcfd77 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:42.847811 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:45:42.878512 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffcbabd-a930-4b1d-976f-197de3f009b8 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:42.884908 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb26964-4b82-4185-bb8e-5d18c0834ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:42.885814 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb26964-4b82-4185-bb8e-5d18c0834ac3 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:42.886274 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb26964-4b82-4185-bb8e-5d18c0834ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:42.886694 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb26964-4b82-4185-bb8e-5d18c0834ac3 admin admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:42.887661 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb26964-4b82-4185-bb8e-5d18c0834ac3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:42.890143 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb26964-4b82-4185-bb8e-5d18c0834ac3 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:42.895892 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9a5c9b-f6c6-4607-b64a-ff952b29f223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:42.896717 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a5c9b-f6c6-4607-b64a-ff952b29f223 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:42.898418 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a5c9b-f6c6-4607-b64a-ff952b29f223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:42.898418 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a5c9b-f6c6-4607-b64a-ff952b29f223 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:42.898418 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9a5c9b-f6c6-4607-b64a-ff952b29f223 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:42.905964 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9a5c9b-f6c6-4607-b64a-ff952b29f223 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:43.027300 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 789/1581] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:45:42 2022] POST /identity/v3/users => generated 303 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:45:43.029174 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb26964-4b82-4185-bb8e-5d18c0834ac3 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:43.042166 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb26964-4b82-4185-bb8e-5d18c0834ac3 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:43.049683 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 793/1582] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:45:42 2022] POST /identity/v3/users => generated 302 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:45:43.050645 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9a5c9b-f6c6-4607-b64a-ff952b29f223 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:43.056953 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237198d8-f61d-4f94-8313-293155d3e0fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:43.057877 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237198d8-f61d-4f94-8313-293155d3e0fa admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:43.058011 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237198d8-f61d-4f94-8313-293155d3e0fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:43.058228 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237198d8-f61d-4f94-8313-293155d3e0fa admin admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:43.058941 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237198d8-f61d-4f94-8313-293155d3e0fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:43.061764 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237198d8-f61d-4f94-8313-293155d3e0fa admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:43.067682 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9a5c9b-f6c6-4607-b64a-ff952b29f223 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:43.078172 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410293fb-d680-48ef-a826-55bd47163d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:43.078933 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410293fb-d680-48ef-a826-55bd47163d3f admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:43.079468 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410293fb-d680-48ef-a826-55bd47163d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:43.079606 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410293fb-d680-48ef-a826-55bd47163d3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:43.084470 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410293fb-d680-48ef-a826-55bd47163d3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:43.084470 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410293fb-d680-48ef-a826-55bd47163d3f admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:43.089709 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:45:43.100093 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 794/1583] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:45:43 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:45:43.100093 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410293fb-d680-48ef-a826-55bd47163d3f admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:43.110728 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410293fb-d680-48ef-a826-55bd47163d3f admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:43.115401 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a1c06-c239-429c-8b74-8ec6f74b68ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:43.116098 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a1c06-c239-429c-8b74-8ec6f74b68ba admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:43.116411 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a1c06-c239-429c-8b74-8ec6f74b68ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:43.117322 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a1c06-c239-429c-8b74-8ec6f74b68ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:43.117885 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8a1c06-c239-429c-8b74-8ec6f74b68ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:43.119553 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8a1c06-c239-429c-8b74-8ec6f74b68ba admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:43.129839 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 795/1584] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:45:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:45:43.131581 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a1c06-c239-429c-8b74-8ec6f74b68ba admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:43.141575 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a1c06-c239-429c-8b74-8ec6f74b68ba admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:43.146684 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbf7489-4532-4fc0-83e5-d4c4ace94b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:43.163980 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbf7489-4532-4fc0-83e5-d4c4ace94b4b admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:43.163980 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbf7489-4532-4fc0-83e5-d4c4ace94b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:43.163980 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbf7489-4532-4fc0-83e5-d4c4ace94b4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:43.163980 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbf7489-4532-4fc0-83e5-d4c4ace94b4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:43.163980 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbf7489-4532-4fc0-83e5-d4c4ace94b4b admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:43.164662 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 796/1585] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:45:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:45:43.165594 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbf7489-4532-4fc0-83e5-d4c4ace94b4b admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:43.175788 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbf7489-4532-4fc0-83e5-d4c4ace94b4b admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:43.180847 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec821d2-2291-415d-8dd1-024e013de913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:43.181514 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec821d2-2291-415d-8dd1-024e013de913 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:43.181801 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec821d2-2291-415d-8dd1-024e013de913 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:43.182079 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec821d2-2291-415d-8dd1-024e013de913 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:43.182774 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec821d2-2291-415d-8dd1-024e013de913 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:43.184760 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec821d2-2291-415d-8dd1-024e013de913 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:43.194403 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 790/1586] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:45:43 2022] POST /identity/v3/users => generated 297 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:45:43.199842 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237198d8-f61d-4f94-8313-293155d3e0fa admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:43.211569 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237198d8-f61d-4f94-8313-293155d3e0fa admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:43.218571 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0af8a58-9418-4576-8fa4-eabd6ef7395e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:43.219435 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af8a58-9418-4576-8fa4-eabd6ef7395e admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:43.219802 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af8a58-9418-4576-8fa4-eabd6ef7395e admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:43.220221 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af8a58-9418-4576-8fa4-eabd6ef7395e admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:43.220934 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0af8a58-9418-4576-8fa4-eabd6ef7395e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:43.222593 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0af8a58-9418-4576-8fa4-eabd6ef7395e admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:43.243172 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 791/1587] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:45:43 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:45:43.243172 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0af8a58-9418-4576-8fa4-eabd6ef7395e admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:43.256962 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0af8a58-9418-4576-8fa4-eabd6ef7395e admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:43.264009 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b60b8-72fc-4292-a365-d72633e173fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:43.264685 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b60b8-72fc-4292-a365-d72633e173fb admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:43.265109 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b60b8-72fc-4292-a365-d72633e173fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:43.265702 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b60b8-72fc-4292-a365-d72633e173fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:43.266919 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235b60b8-72fc-4292-a365-d72633e173fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:43.269886 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235b60b8-72fc-4292-a365-d72633e173fb admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:43.290221 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 792/1588] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:45:43 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:45:43.290221 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b60b8-72fc-4292-a365-d72633e173fb admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:43.305334 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 797/1589] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:45:43 2022] POST /identity/v3/users => generated 288 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:45:43.308360 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec821d2-2291-415d-8dd1-024e013de913 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:43.308483 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b60b8-72fc-4292-a365-d72633e173fb admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:43.314943 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32daf55a-b8b5-4900-bba6-963344a552dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:43.315919 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32daf55a-b8b5-4900-bba6-963344a552dc admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:43.316484 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32daf55a-b8b5-4900-bba6-963344a552dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:43.316876 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32daf55a-b8b5-4900-bba6-963344a552dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:43.318344 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32daf55a-b8b5-4900-bba6-963344a552dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:43.318472 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec821d2-2291-415d-8dd1-024e013de913 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:43.321006 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32daf55a-b8b5-4900-bba6-963344a552dc admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:43.324878 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a946be-a863-438b-98c5-fcfd54cc5932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:43.325760 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a946be-a863-438b-98c5-fcfd54cc5932 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:43.326287 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a946be-a863-438b-98c5-fcfd54cc5932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:43.326814 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a946be-a863-438b-98c5-fcfd54cc5932 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:43.327861 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a946be-a863-438b-98c5-fcfd54cc5932 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:43.329927 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a946be-a863-438b-98c5-fcfd54cc5932 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:43.337774 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 793/1590] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:45:43 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:45:43.343802 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32daf55a-b8b5-4900-bba6-963344a552dc admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:43.346151 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 798/1591] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:45:43 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:45:43.347379 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a946be-a863-438b-98c5-fcfd54cc5932 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:43.358767 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32daf55a-b8b5-4900-bba6-963344a552dc admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:43.359371 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a946be-a863-438b-98c5-fcfd54cc5932 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:43.364556 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5380ede4-e225-4532-bc88-28b44c992ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:43.365234 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda767cd-5053-415e-b08f-0fc2bb0a49f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:43.365424 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380ede4-e225-4532-bc88-28b44c992ba1 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:43.365788 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380ede4-e225-4532-bc88-28b44c992ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:43.365946 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda767cd-5053-415e-b08f-0fc2bb0a49f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:43.366192 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda767cd-5053-415e-b08f-0fc2bb0a49f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:43.366336 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380ede4-e225-4532-bc88-28b44c992ba1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:43.366540 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda767cd-5053-415e-b08f-0fc2bb0a49f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:43.368015 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda767cd-5053-415e-b08f-0fc2bb0a49f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:43.368680 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5380ede4-e225-4532-bc88-28b44c992ba1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:43.369641 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda767cd-5053-415e-b08f-0fc2bb0a49f9 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:43.370469 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5380ede4-e225-4532-bc88-28b44c992ba1 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:43.380980 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 794/1592] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:45:43 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:45:43.382361 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda767cd-5053-415e-b08f-0fc2bb0a49f9 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:43.384085 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 799/1593] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:45:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:45:43.385278 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5380ede4-e225-4532-bc88-28b44c992ba1 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:43.394723 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda767cd-5053-415e-b08f-0fc2bb0a49f9 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:43.396433 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5380ede4-e225-4532-bc88-28b44c992ba1 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:43.402385 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0d309b-d8b3-40f4-ab35-2702e2c115dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:43.402699 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d309b-d8b3-40f4-ab35-2702e2c115dc admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:43.403004 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d309b-d8b3-40f4-ab35-2702e2c115dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:43.403245 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d309b-d8b3-40f4-ab35-2702e2c115dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:43.403780 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5336e1a-2d0d-413e-aeab-3c3c8e14ada0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:43.405141 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0d309b-d8b3-40f4-ab35-2702e2c115dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:43.405248 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5336e1a-2d0d-413e-aeab-3c3c8e14ada0 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:43.405248 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5336e1a-2d0d-413e-aeab-3c3c8e14ada0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:43.405248 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5336e1a-2d0d-413e-aeab-3c3c8e14ada0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:43.405816 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0d309b-d8b3-40f4-ab35-2702e2c115dc admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:43.406333 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5336e1a-2d0d-413e-aeab-3c3c8e14ada0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:43.408760 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5336e1a-2d0d-413e-aeab-3c3c8e14ada0 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:43.424070 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 800/1594] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:45:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:45:43.425335 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0d309b-d8b3-40f4-ab35-2702e2c115dc admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:43.432650 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 795/1595] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:45:43 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:45:43.432650 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5336e1a-2d0d-413e-aeab-3c3c8e14ada0 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:43.437553 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0d309b-d8b3-40f4-ab35-2702e2c115dc admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:43.443601 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5336e1a-2d0d-413e-aeab-3c3c8e14ada0 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:43.444750 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1919c60f-abe8-46e8-b65b-2f6863b640ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:43.446242 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919c60f-abe8-46e8-b65b-2f6863b640ad admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:43.446618 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919c60f-abe8-46e8-b65b-2f6863b640ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:43.447012 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919c60f-abe8-46e8-b65b-2f6863b640ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:43.447882 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1919c60f-abe8-46e8-b65b-2f6863b640ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:43.449161 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209178c9-efe4-42f9-bb4b-09542e60e4bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:43.450120 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209178c9-efe4-42f9-bb4b-09542e60e4bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:43.450405 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209178c9-efe4-42f9-bb4b-09542e60e4bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:43.450692 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209178c9-efe4-42f9-bb4b-09542e60e4bd admin admin] PATH_INFO: `/v3/projects/32b83bc62e8b44639f7034aa9b269f3e/users/3e44347ae1ad434ba9d1f22945125b6d/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:43.451169 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1919c60f-abe8-46e8-b65b-2f6863b640ad admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:43.474110 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209178c9-efe4-42f9-bb4b-09542e60e4bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=32b83bc62e8b44639f7034aa9b269f3e, user_id=3e44347ae1ad434ba9d1f22945125b6d, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:43.474317 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 801/1596] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:45:43 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:45:43.475877 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1919c60f-abe8-46e8-b65b-2f6863b640ad admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:43.479416 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209178c9-efe4-42f9-bb4b-09542e60e4bd admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:43.491640 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1919c60f-abe8-46e8-b65b-2f6863b640ad admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:43.498689 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f5147-ad87-4ebb-b177-b031621db133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:43.500161 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f5147-ad87-4ebb-b177-b031621db133 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:43.501159 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f5147-ad87-4ebb-b177-b031621db133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:43.501159 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f5147-ad87-4ebb-b177-b031621db133 admin admin] PATH_INFO: `/v3/projects/1893db2223fc4395a202c5124c94fdf8/users/d6cfe76f2e98474f802171dbf7a61244/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:43.531175 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5f5147-ad87-4ebb-b177-b031621db133 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1893db2223fc4395a202c5124c94fdf8, user_id=d6cfe76f2e98474f802171dbf7a61244, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:43.532737 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: WARNING py.warnings [None req-1b5f5147-ad87-4ebb-b177-b031621db133 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 16:45:43.532737 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: warnings.warn(msg) Jul 18 16:45:43.532737 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: Jul 18 16:45:43.534598 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5f5147-ad87-4ebb-b177-b031621db133 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:43.571523 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 796/1597] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:45:43 2022] PUT /identity/v3/projects/32b83bc62e8b44639f7034aa9b269f3e/users/3e44347ae1ad434ba9d1f22945125b6d/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:45:43.583160 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209178c9-efe4-42f9-bb4b-09542e60e4bd admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:43.659337 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209178c9-efe4-42f9-bb4b-09542e60e4bd admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:43.668908 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 802/1598] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:45:43 2022] PUT /identity/v3/projects/1893db2223fc4395a202c5124c94fdf8/users/d6cfe76f2e98474f802171dbf7a61244/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:45:43.669980 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f5147-ad87-4ebb-b177-b031621db133 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:43.701831 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:45:43.744843 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3f1cc0-70ac-4c1d-a044-bf190bd15de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:43.745708 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f1cc0-70ac-4c1d-a044-bf190bd15de7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:43.746037 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f1cc0-70ac-4c1d-a044-bf190bd15de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:43.746308 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f1cc0-70ac-4c1d-a044-bf190bd15de7 admin admin] PATH_INFO: `/v3/projects/47f8e892e9284e2f926d1a23b80f0a5c/users/0a36d1f2e0b1457b81946ed9af4c05b0/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:43.756191 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f5147-ad87-4ebb-b177-b031621db133 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:43.761835 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caa7c0c-4a27-47de-b338-85f2da4a7673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:43.762559 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caa7c0c-4a27-47de-b338-85f2da4a7673 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:43.762800 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caa7c0c-4a27-47de-b338-85f2da4a7673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:43.763199 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caa7c0c-4a27-47de-b338-85f2da4a7673 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:43.763978 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caa7c0c-4a27-47de-b338-85f2da4a7673 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:43.764802 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3f1cc0-70ac-4c1d-a044-bf190bd15de7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47f8e892e9284e2f926d1a23b80f0a5c, user_id=0a36d1f2e0b1457b81946ed9af4c05b0, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:43.765940 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caa7c0c-4a27-47de-b338-85f2da4a7673 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:43.767471 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3f1cc0-70ac-4c1d-a044-bf190bd15de7 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:43.777465 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 803/1599] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:45:43 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:45:43.777465 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caa7c0c-4a27-47de-b338-85f2da4a7673 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:43.791783 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caa7c0c-4a27-47de-b338-85f2da4a7673 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:43.797082 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561cf63d-7ca2-4821-8926-aed2e4a37507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:43.798200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561cf63d-7ca2-4821-8926-aed2e4a37507 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:43.798490 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561cf63d-7ca2-4821-8926-aed2e4a37507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:43.798773 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561cf63d-7ca2-4821-8926-aed2e4a37507 admin admin] PATH_INFO: `/v3/projects/4bcee7e01003431db156174be70a8577/users/f2ddfbaed6c14218a89ea9d90e0b4ae8/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:43.819375 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561cf63d-7ca2-4821-8926-aed2e4a37507 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bcee7e01003431db156174be70a8577, user_id=f2ddfbaed6c14218a89ea9d90e0b4ae8, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:43.822084 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561cf63d-7ca2-4821-8926-aed2e4a37507 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:43.962025 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 797/1600] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:45:43 2022] PUT /identity/v3/projects/47f8e892e9284e2f926d1a23b80f0a5c/users/0a36d1f2e0b1457b81946ed9af4c05b0/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:45:43.963128 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3f1cc0-70ac-4c1d-a044-bf190bd15de7 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:44.017510 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 804/1601] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:45:43 2022] PUT /identity/v3/projects/4bcee7e01003431db156174be70a8577/users/f2ddfbaed6c14218a89ea9d90e0b4ae8/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:45:44.019263 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561cf63d-7ca2-4821-8926-aed2e4a37507 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:44.035308 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3f1cc0-70ac-4c1d-a044-bf190bd15de7 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:44.064683 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:45:44.124763 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561cf63d-7ca2-4821-8926-aed2e4a37507 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:44.133632 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3850a808-c158-450b-a02f-04519272a3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:44.134373 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3850a808-c158-450b-a02f-04519272a3d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:44.134694 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3850a808-c158-450b-a02f-04519272a3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:44.135204 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3850a808-c158-450b-a02f-04519272a3d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:44.135905 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7005dab2-81c4-42ca-a29a-3808bd662ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:44.136203 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3850a808-c158-450b-a02f-04519272a3d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:44.137008 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7005dab2-81c4-42ca-a29a-3808bd662ef7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:44.137282 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7005dab2-81c4-42ca-a29a-3808bd662ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:44.137739 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7005dab2-81c4-42ca-a29a-3808bd662ef7 admin admin] PATH_INFO: `/v3/projects/2ca139607f5147ebb9acd9a03931f2f8/users/7852764385e34367936225432d9846cd/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:44.137884 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3850a808-c158-450b-a02f-04519272a3d5 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:44.158943 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 805/1602] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:45:44.159379 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261213a9-ada6-41d3-8554-9ca6885607fc None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:44.159620 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261213a9-ada6-41d3-8554-9ca6885607fc None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:44.159978 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261213a9-ada6-41d3-8554-9ca6885607fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:44.303260 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7005dab2-81c4-42ca-a29a-3808bd662ef7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ca139607f5147ebb9acd9a03931f2f8, user_id=7852764385e34367936225432d9846cd, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:44.303260 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7005dab2-81c4-42ca-a29a-3808bd662ef7 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:44.305435 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-261213a9-ada6-41d3-8554-9ca6885607fc None None] MFA Rules not processed for user `3e44347ae1ad434ba9d1f22945125b6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:44.305435 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-261213a9-ada6-41d3-8554-9ca6885607fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:44.343024 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 806/1603] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:45:44 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:45:44.344536 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab9e8f-5888-4cd9-92cb-b8f8e3835e83 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:44.345019 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab9e8f-5888-4cd9-92cb-b8f8e3835e83 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:44.345397 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab9e8f-5888-4cd9-92cb-b8f8e3835e83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:44.392283 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-e2ab9e8f-5888-4cd9-92cb-b8f8e3835e83 None None] MFA Rules not processed for user `d6cfe76f2e98474f802171dbf7a61244`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:44.393614 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e2ab9e8f-5888-4cd9-92cb-b8f8e3835e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:44.408420 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 798/1604] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:45:43 2022] PUT /identity/v3/projects/2ca139607f5147ebb9acd9a03931f2f8/users/7852764385e34367936225432d9846cd/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:45:44.410878 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7005dab2-81c4-42ca-a29a-3808bd662ef7 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:44.465062 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:45:44.472120 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 807/1605] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:45:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:45:44.476115 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f036d9db-f558-42e2-9b2d-3451a4aefb0c None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:44.480715 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f036d9db-f558-42e2-9b2d-3451a4aefb0c None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:44.480715 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f036d9db-f558-42e2-9b2d-3451a4aefb0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:44.517651 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7005dab2-81c4-42ca-a29a-3808bd662ef7 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:44.518490 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-f036d9db-f558-42e2-9b2d-3451a4aefb0c None None] MFA Rules not processed for user `0a36d1f2e0b1457b81946ed9af4c05b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:44.520218 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f036d9db-f558-42e2-9b2d-3451a4aefb0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:44.526222 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b618046-c2b9-4e0d-aaaa-796127510f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:44.528299 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b618046-c2b9-4e0d-aaaa-796127510f6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:44.531844 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b618046-c2b9-4e0d-aaaa-796127510f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:44.531844 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b618046-c2b9-4e0d-aaaa-796127510f6b admin admin] PATH_INFO: `/v3/projects/69c48eddc2334918970e61725bedafd1/users/642ba7eed36e4956834c06b771259a42/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:44.564037 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:45:44.564380 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b618046-c2b9-4e0d-aaaa-796127510f6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=69c48eddc2334918970e61725bedafd1, user_id=642ba7eed36e4956834c06b771259a42, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:44.568835 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b618046-c2b9-4e0d-aaaa-796127510f6b admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:44.611405 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 808/1606] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:45:44 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:45:44.611405 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54b1a2-1e8f-432f-a922-1c8071ee421d None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:44.611405 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54b1a2-1e8f-432f-a922-1c8071ee421d None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:44.611405 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54b1a2-1e8f-432f-a922-1c8071ee421d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:44.660169 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 799/1607] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:45:44 2022] PUT /identity/v3/projects/69c48eddc2334918970e61725bedafd1/users/642ba7eed36e4956834c06b771259a42/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:45:44.661120 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b618046-c2b9-4e0d-aaaa-796127510f6b admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:44.661499 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-fe54b1a2-1e8f-432f-a922-1c8071ee421d None None] MFA Rules not processed for user `f2ddfbaed6c14218a89ea9d90e0b4ae8`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:44.662942 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-fe54b1a2-1e8f-432f-a922-1c8071ee421d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:44.753017 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b618046-c2b9-4e0d-aaaa-796127510f6b admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:44.757900 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 809/1608] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:45:44 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:45:44.757900 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe54b1a2-1e8f-432f-a922-1c8071ee421d None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:44.765425 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1d7e83-d954-407d-8ca1-dfccd2ee271d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:44.765425 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d7e83-d954-407d-8ca1-dfccd2ee271d admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:44.765425 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d7e83-d954-407d-8ca1-dfccd2ee271d admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:44.765425 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d7e83-d954-407d-8ca1-dfccd2ee271d admin admin] PATH_INFO: `/v3/projects/9e65f14a4bef42df904a5f15befe2e16/users/c243a50903b244519ccd58f02cb798da/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:44.767792 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe54b1a2-1e8f-432f-a922-1c8071ee421d None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:44.777423 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0487b09c-49bd-4e8d-86ee-c73db68ce319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:44.778638 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0487b09c-49bd-4e8d-86ee-c73db68ce319 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:44.778940 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0487b09c-49bd-4e8d-86ee-c73db68ce319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:44.779237 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0487b09c-49bd-4e8d-86ee-c73db68ce319 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:44.780005 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0487b09c-49bd-4e8d-86ee-c73db68ce319 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:44.781940 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1d7e83-d954-407d-8ca1-dfccd2ee271d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e65f14a4bef42df904a5f15befe2e16, user_id=c243a50903b244519ccd58f02cb798da, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:44.782627 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0487b09c-49bd-4e8d-86ee-c73db68ce319 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:44.787532 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1d7e83-d954-407d-8ca1-dfccd2ee271d admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:44.900918 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 800/1609] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:45:44 2022] PUT /identity/v3/projects/9e65f14a4bef42df904a5f15befe2e16/users/c243a50903b244519ccd58f02cb798da/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:45:44.900918 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be885e-d783-4f46-9bb1-d3ddb0a06e9f None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:44.900918 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be885e-d783-4f46-9bb1-d3ddb0a06e9f None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:44.900918 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be885e-d783-4f46-9bb1-d3ddb0a06e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:44.906092 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 810/1610] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:45:44 2022] POST /identity/v3/projects => generated 357 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:45:44.908042 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8929d4-1993-48a6-94a7-38f7596d02a6 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:44.908670 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8929d4-1993-48a6-94a7-38f7596d02a6 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:44.909293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8929d4-1993-48a6-94a7-38f7596d02a6 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:44.917733 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 811/1611] 10.222.0.26 () {54 vars in 962 bytes} [Mon Jul 18 16:45:44 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:45:44.919524 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145313b5-c038-48e0-8d25-9aff28f55837 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:44.920140 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145313b5-c038-48e0-8d25-9aff28f55837 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:44.920432 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145313b5-c038-48e0-8d25-9aff28f55837 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:44.922015 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 812/1612] 10.222.0.26 () {54 vars in 962 bytes} [Mon Jul 18 16:45:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:45:44.930339 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-d8be885e-d783-4f46-9bb1-d3ddb0a06e9f None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:44.932096 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b8b440-55eb-48c1-91e8-57449936ada2 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:44.932096 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b8b440-55eb-48c1-91e8-57449936ada2 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:44.932096 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b8b440-55eb-48c1-91e8-57449936ada2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:44.932408 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-d8be885e-d783-4f46-9bb1-d3ddb0a06e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:44.961477 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-89b8b440-55eb-48c1-91e8-57449936ada2 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:44.963430 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-89b8b440-55eb-48c1-91e8-57449936ada2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:45.037937 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 801/1613] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:45:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:45:45.039084 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8be885e-d783-4f46-9bb1-d3ddb0a06e9f None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:45.058331 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8be885e-d783-4f46-9bb1-d3ddb0a06e9f None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:45.064093 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 813/1614] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:45:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:45:45.064093 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deb8fda-13f0-4a5f-89f8-a822058c31ab None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:45.064093 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deb8fda-13f0-4a5f-89f8-a822058c31ab None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:45.064093 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deb8fda-13f0-4a5f-89f8-a822058c31ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:45.067204 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7382fb-4095-4647-b0a0-d06d41aadff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:45.068653 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7382fb-4095-4647-b0a0-d06d41aadff0 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:45.068879 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7382fb-4095-4647-b0a0-d06d41aadff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:45.069252 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7382fb-4095-4647-b0a0-d06d41aadff0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:45.074481 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7382fb-4095-4647-b0a0-d06d41aadff0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:45.078408 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7382fb-4095-4647-b0a0-d06d41aadff0 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:45.089980 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-5deb8fda-13f0-4a5f-89f8-a822058c31ab None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:45.091205 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-5deb8fda-13f0-4a5f-89f8-a822058c31ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:45.102482 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 814/1615] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:45:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:45:45.103407 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deb8fda-13f0-4a5f-89f8-a822058c31ab None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:45.115578 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deb8fda-13f0-4a5f-89f8-a822058c31ab None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:45.122403 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9167541-9705-41e7-ab88-8f0ffc778fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:45.123163 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9167541-9705-41e7-ab88-8f0ffc778fc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:45.123430 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9167541-9705-41e7-ab88-8f0ffc778fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:45.123738 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9167541-9705-41e7-ab88-8f0ffc778fc1 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:45.124440 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9167541-9705-41e7-ab88-8f0ffc778fc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:45.126776 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9167541-9705-41e7-ab88-8f0ffc778fc1 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:45.190650 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 802/1616] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:45:45 2022] POST /identity/v3/projects => generated 405 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:45:45.194309 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d52ed3-0ea8-4da1-8e9f-f337145a38b8 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:45.196191 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d52ed3-0ea8-4da1-8e9f-f337145a38b8 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:45.197382 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d52ed3-0ea8-4da1-8e9f-f337145a38b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:45.245507 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 815/1617] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:45:45 2022] POST /identity/v3/users => generated 285 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:45:45.248653 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb691f-b393-4f0c-a0c4-72d7dff95f22 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:45.249160 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb691f-b393-4f0c-a0c4-72d7dff95f22 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:45.249619 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb691f-b393-4f0c-a0c4-72d7dff95f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:45.278609 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-a5d52ed3-0ea8-4da1-8e9f-f337145a38b8 None None] MFA Rules not processed for user `5b868bfa44b94ca5b470166d39f20e54`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:45.281984 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-a5d52ed3-0ea8-4da1-8e9f-f337145a38b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:45.289963 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-8aeb691f-b393-4f0c-a0c4-72d7dff95f22 None None] MFA Rules not processed for user `5b868bfa44b94ca5b470166d39f20e54`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:45.292802 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-8aeb691f-b393-4f0c-a0c4-72d7dff95f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:45.312666 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:45:45.318921 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:45:45.412383 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 803/1618] 10.222.0.26 () {58 vars in 1059 bytes} [Mon Jul 18 16:45:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:45:45.413885 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 816/1619] 10.222.0.26 () {58 vars in 1059 bytes} [Mon Jul 18 16:45:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:45:45.416421 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeb691f-b393-4f0c-a0c4-72d7dff95f22 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:45.421716 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d52ed3-0ea8-4da1-8e9f-f337145a38b8 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:45.522393 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d52ed3-0ea8-4da1-8e9f-f337145a38b8 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:45.546573 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77c4d26-5004-472b-9659-f1b8c0318a61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:45.550761 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c4d26-5004-472b-9659-f1b8c0318a61 service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:45.552922 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c4d26-5004-472b-9659-f1b8c0318a61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:45.552922 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c4d26-5004-472b-9659-f1b8c0318a61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:45.578538 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeb691f-b393-4f0c-a0c4-72d7dff95f22 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:45.598626 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e074226-ad20-4a20-bea0-59cff3a0101b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:45.599899 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e074226-ad20-4a20-bea0-59cff3a0101b service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:45.600416 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e074226-ad20-4a20-bea0-59cff3a0101b service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:45.600966 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e074226-ad20-4a20-bea0-59cff3a0101b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:45.729319 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77c4d26-5004-472b-9659-f1b8c0318a61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:45.731146 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77c4d26-5004-472b-9659-f1b8c0318a61 service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:45.747164 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 804/1620] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:45:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:45:45.748431 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77c4d26-5004-472b-9659-f1b8c0318a61 service neutron] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:45.758259 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77c4d26-5004-472b-9659-f1b8c0318a61 service neutron] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:45.764699 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed36fda-23d9-42c6-b962-23075ea97081 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:45.767186 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed36fda-23d9-42c6-b962-23075ea97081 service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:45.768210 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed36fda-23d9-42c6-b962-23075ea97081 service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:45.768707 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed36fda-23d9-42c6-b962-23075ea97081 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:45.781464 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed36fda-23d9-42c6-b962-23075ea97081 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:45.783426 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed36fda-23d9-42c6-b962-23075ea97081 service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:45.785846 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e074226-ad20-4a20-bea0-59cff3a0101b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:45.787418 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e074226-ad20-4a20-bea0-59cff3a0101b service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:45.809430 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 805/1621] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:45:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:45:45.811554 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed36fda-23d9-42c6-b962-23075ea97081 service neutron] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:45.821939 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 817/1622] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:45:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:45:45.824902 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e074226-ad20-4a20-bea0-59cff3a0101b service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:45.843737 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed36fda-23d9-42c6-b962-23075ea97081 service neutron] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:45.852722 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e074226-ad20-4a20-bea0-59cff3a0101b service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:45.855301 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9ca2f1-0750-49e1-87ac-a332c145bb6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:45.856399 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ca2f1-0750-49e1-87ac-a332c145bb6f admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:45.856891 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ca2f1-0750-49e1-87ac-a332c145bb6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:45.858428 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ca2f1-0750-49e1-87ac-a332c145bb6f admin admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:45.860322 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9ca2f1-0750-49e1-87ac-a332c145bb6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:45.866485 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9ca2f1-0750-49e1-87ac-a332c145bb6f admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:45.874559 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81671a5-8892-4f63-bae1-2cba5e2969cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:45.875646 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81671a5-8892-4f63-bae1-2cba5e2969cf admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:45.876031 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81671a5-8892-4f63-bae1-2cba5e2969cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:45.876314 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81671a5-8892-4f63-bae1-2cba5e2969cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:45.877767 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81671a5-8892-4f63-bae1-2cba5e2969cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:45.880288 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81671a5-8892-4f63-bae1-2cba5e2969cf admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:45.895340 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 818/1623] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:45:45 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:45:45.900747 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81671a5-8892-4f63-bae1-2cba5e2969cf admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:45.936850 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81671a5-8892-4f63-bae1-2cba5e2969cf admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:45.944701 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e29fc2-9ebe-4159-b95c-a314032a02d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:45.946527 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e29fc2-9ebe-4159-b95c-a314032a02d9 service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:45.947269 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e29fc2-9ebe-4159-b95c-a314032a02d9 service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:45.947269 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e29fc2-9ebe-4159-b95c-a314032a02d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:45.971614 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 806/1624] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:45:45 2022] POST /identity/v3/users => generated 309 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:45:45.972731 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9ca2f1-0750-49e1-87ac-a332c145bb6f admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:46.003081 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9ca2f1-0750-49e1-87ac-a332c145bb6f admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:46.011583 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcffb766-c17b-44db-99df-ed8b9fbb5930 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:46.013186 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcffb766-c17b-44db-99df-ed8b9fbb5930 service nova] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:46.013796 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcffb766-c17b-44db-99df-ed8b9fbb5930 service nova] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:46.014471 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcffb766-c17b-44db-99df-ed8b9fbb5930 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:46.044469 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:45:46.048718 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e29fc2-9ebe-4159-b95c-a314032a02d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:46.050508 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e29fc2-9ebe-4159-b95c-a314032a02d9 service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:46.062307 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 819/1625] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:45:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:45:46.063163 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e29fc2-9ebe-4159-b95c-a314032a02d9 service nova] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:46.075531 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e29fc2-9ebe-4159-b95c-a314032a02d9 service nova] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:46.081933 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b598bff7-e778-4b4a-831f-f1b33fa54d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:46.083505 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b598bff7-e778-4b4a-831f-f1b33fa54d8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:46.088296 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b598bff7-e778-4b4a-831f-f1b33fa54d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:46.088296 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b598bff7-e778-4b4a-831f-f1b33fa54d8e admin admin] PATH_INFO: `/v3/projects/50eba0987a1d4aa0b741d7bcac564c38/users/1e37d08c5c594a9abf7e8c2b6f2f7432/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:46.096429 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcffb766-c17b-44db-99df-ed8b9fbb5930 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:46.099781 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcffb766-c17b-44db-99df-ed8b9fbb5930 service nova] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:46.106293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b598bff7-e778-4b4a-831f-f1b33fa54d8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=50eba0987a1d4aa0b741d7bcac564c38, user_id=1e37d08c5c594a9abf7e8c2b6f2f7432, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:46.109815 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b598bff7-e778-4b4a-831f-f1b33fa54d8e admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:46.114219 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 807/1626] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:45:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:45:46.115574 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcffb766-c17b-44db-99df-ed8b9fbb5930 service nova] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:46.127710 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcffb766-c17b-44db-99df-ed8b9fbb5930 service nova] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:46.134984 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37fb55c-e2d6-429b-a622-833b61b9b57b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:46.136004 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37fb55c-e2d6-429b-a622-833b61b9b57b admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:46.136495 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37fb55c-e2d6-429b-a622-833b61b9b57b admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:46.137040 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37fb55c-e2d6-429b-a622-833b61b9b57b admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:46.138013 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37fb55c-e2d6-429b-a622-833b61b9b57b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:46.140336 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37fb55c-e2d6-429b-a622-833b61b9b57b admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:46.153788 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 808/1627] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:45:46 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:45:46.157989 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37fb55c-e2d6-429b-a622-833b61b9b57b admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:46.173974 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37fb55c-e2d6-429b-a622-833b61b9b57b admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:46.186194 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c291eb-d27c-47b6-a783-52f26ef81853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:46.187720 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c291eb-d27c-47b6-a783-52f26ef81853 admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:46.188205 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c291eb-d27c-47b6-a783-52f26ef81853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:46.188681 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c291eb-d27c-47b6-a783-52f26ef81853 admin admin] PATH_INFO: `/v3/projects/1931dfb579c54941b9ea02300866be98/users/5cae474a74e44b08a3b62d1877e0e40d/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:46.214929 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c291eb-d27c-47b6-a783-52f26ef81853 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1931dfb579c54941b9ea02300866be98, user_id=5cae474a74e44b08a3b62d1877e0e40d, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:46.218921 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c291eb-d27c-47b6-a783-52f26ef81853 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:46.239157 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 820/1628] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:45:46 2022] PUT /identity/v3/projects/50eba0987a1d4aa0b741d7bcac564c38/users/1e37d08c5c594a9abf7e8c2b6f2f7432/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:45:46.243682 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b598bff7-e778-4b4a-831f-f1b33fa54d8e admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:46.322286 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b598bff7-e778-4b4a-831f-f1b33fa54d8e admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:46.328025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b81ccd7-dbea-475a-b041-504d5ff54773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:46.328665 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b81ccd7-dbea-475a-b041-504d5ff54773 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:46.328967 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b81ccd7-dbea-475a-b041-504d5ff54773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:46.329256 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b81ccd7-dbea-475a-b041-504d5ff54773 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:46.782790 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b81ccd7-dbea-475a-b041-504d5ff54773 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:46.782790 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b81ccd7-dbea-475a-b041-504d5ff54773 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:46.782790 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 821/1629] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:45:46 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:45:46.782790 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b81ccd7-dbea-475a-b041-504d5ff54773 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:46.782790 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b81ccd7-dbea-475a-b041-504d5ff54773 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:46.782790 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:45:46.785863 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 809/1630] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:45:46 2022] PUT /identity/v3/projects/1931dfb579c54941b9ea02300866be98/users/5cae474a74e44b08a3b62d1877e0e40d/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:45:46.785863 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c291eb-d27c-47b6-a783-52f26ef81853 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:46.785863 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c291eb-d27c-47b6-a783-52f26ef81853 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:46.785863 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7336fcc-5e76-4b15-9f6a-478edd8b6ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:46.797817 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7336fcc-5e76-4b15-9f6a-478edd8b6ac9 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:46.797817 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7336fcc-5e76-4b15-9f6a-478edd8b6ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:46.797817 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7336fcc-5e76-4b15-9f6a-478edd8b6ac9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:46.797817 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7336fcc-5e76-4b15-9f6a-478edd8b6ac9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:46.797817 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7336fcc-5e76-4b15-9f6a-478edd8b6ac9 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:46.797817 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 810/1631] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:45:46 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:45:46.813763 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7336fcc-5e76-4b15-9f6a-478edd8b6ac9 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:46.813763 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7336fcc-5e76-4b15-9f6a-478edd8b6ac9 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:46.831215 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e790455-e710-49d3-b878-3b8105691ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:46.831215 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e790455-e710-49d3-b878-3b8105691ca8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:46.831215 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e790455-e710-49d3-b878-3b8105691ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:46.831215 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e790455-e710-49d3-b878-3b8105691ca8 admin admin] PATH_INFO: `/v3/projects/50eba0987a1d4aa0b741d7bcac564c38/users/1e37d08c5c594a9abf7e8c2b6f2f7432/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:46.837785 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70185079-0e5b-4f14-9150-91cdfe34712a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:46.837785 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70185079-0e5b-4f14-9150-91cdfe34712a admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:46.837785 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70185079-0e5b-4f14-9150-91cdfe34712a admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:46.837785 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70185079-0e5b-4f14-9150-91cdfe34712a admin admin] PATH_INFO: `/v3/projects/1931dfb579c54941b9ea02300866be98/users/5cae474a74e44b08a3b62d1877e0e40d/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:46.839631 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70185079-0e5b-4f14-9150-91cdfe34712a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1931dfb579c54941b9ea02300866be98, user_id=5cae474a74e44b08a3b62d1877e0e40d, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:46.839631 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70185079-0e5b-4f14-9150-91cdfe34712a admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:46.843827 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e790455-e710-49d3-b878-3b8105691ca8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50eba0987a1d4aa0b741d7bcac564c38, user_id=1e37d08c5c594a9abf7e8c2b6f2f7432, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:46.843827 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e790455-e710-49d3-b878-3b8105691ca8 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:46.843827 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 822/1632] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:45:46 2022] PUT /identity/v3/projects/50eba0987a1d4aa0b741d7bcac564c38/users/1e37d08c5c594a9abf7e8c2b6f2f7432/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:45:46.843827 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d390d-5490-4f01-9589-8920381226e4 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:46.843827 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d390d-5490-4f01-9589-8920381226e4 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:46.843827 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d390d-5490-4f01-9589-8920381226e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:46.852613 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 811/1633] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:45:46 2022] PUT /identity/v3/projects/1931dfb579c54941b9ea02300866be98/users/5cae474a74e44b08a3b62d1877e0e40d/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:45:46.852613 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a6f3ed-ec6e-4177-a4e9-415e9fc753ac None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:46.852613 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a6f3ed-ec6e-4177-a4e9-415e9fc753ac None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:46.852613 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a6f3ed-ec6e-4177-a4e9-415e9fc753ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:46.855310 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-8f0d390d-5490-4f01-9589-8920381226e4 None None] MFA Rules not processed for user `1e37d08c5c594a9abf7e8c2b6f2f7432`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:46.855310 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-8f0d390d-5490-4f01-9589-8920381226e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:46.857358 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-01a6f3ed-ec6e-4177-a4e9-415e9fc753ac None None] MFA Rules not processed for user `5cae474a74e44b08a3b62d1877e0e40d`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:46.857358 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-01a6f3ed-ec6e-4177-a4e9-415e9fc753ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:46.857358 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 812/1634] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:45:46 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:45:46.857358 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191c18e-49f5-4fea-852a-861e8c3711b0 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:46.857358 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191c18e-49f5-4fea-852a-861e8c3711b0 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:46.857358 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191c18e-49f5-4fea-852a-861e8c3711b0 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:46.860347 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 813/1635] 10.222.0.26 () {54 vars in 912 bytes} [Mon Jul 18 16:45:46 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:45:46.860347 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3191c18e-49f5-4fea-852a-861e8c3711b0 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:46.869551 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 823/1636] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:45:46 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:45:46.873040 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619932c-362e-4c99-be59-b7c44c455bd1 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:46.873346 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619932c-362e-4c99-be59-b7c44c455bd1 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:46.881120 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619932c-362e-4c99-be59-b7c44c455bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:46.896892 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-8619932c-362e-4c99-be59-b7c44c455bd1 None None] MFA Rules not processed for user `5b868bfa44b94ca5b470166d39f20e54`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:46.898691 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-8619932c-362e-4c99-be59-b7c44c455bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:46.913396 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3191c18e-49f5-4fea-852a-861e8c3711b0 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:46.920096 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7960664a-50ed-4750-829b-e71719239143 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:46.921399 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7960664a-50ed-4750-829b-e71719239143 service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:46.921399 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7960664a-50ed-4750-829b-e71719239143 service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:46.921674 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7960664a-50ed-4750-829b-e71719239143 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:46.937017 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7960664a-50ed-4750-829b-e71719239143 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:46.939029 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7960664a-50ed-4750-829b-e71719239143 service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:46.953821 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 814/1637] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:45:46 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:45:46.956700 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ec1480-c87b-497e-9c5e-c2b9769a2206 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:46.957181 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ec1480-c87b-497e-9c5e-c2b9769a2206 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:46.957624 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ec1480-c87b-497e-9c5e-c2b9769a2206 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:46.960066 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 815/1638] 10.222.0.26 () {54 vars in 912 bytes} [Mon Jul 18 16:45:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:45:46.961728 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ec1480-c87b-497e-9c5e-c2b9769a2206 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:46.977961 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ec1480-c87b-497e-9c5e-c2b9769a2206 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:46.991216 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f863dd9a-705b-4772-8931-41eed21d7599 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:46.991216 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f863dd9a-705b-4772-8931-41eed21d7599 service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:46.991216 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f863dd9a-705b-4772-8931-41eed21d7599 service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:46.991216 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f863dd9a-705b-4772-8931-41eed21d7599 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:46.996246 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 824/1639] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jul 18 16:45:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:45:46.997649 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e9abde-ec8e-459f-8997-54d8eb081451 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:46.997942 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e9abde-ec8e-459f-8997-54d8eb081451 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:46.998222 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e9abde-ec8e-459f-8997-54d8eb081451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:47.003365 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f863dd9a-705b-4772-8931-41eed21d7599 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:47.005453 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f863dd9a-705b-4772-8931-41eed21d7599 service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:47.035921 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-f2e9abde-ec8e-459f-8997-54d8eb081451 None None] MFA Rules not processed for user `5b868bfa44b94ca5b470166d39f20e54`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:47.037220 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f2e9abde-ec8e-459f-8997-54d8eb081451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:47.043677 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 816/1640] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:45:46 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:45:47.065545 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 825/1641] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jul 18 16:45:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:45:47.077376 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f863dd9a-705b-4772-8931-41eed21d7599 service cinder] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:47.087770 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f863dd9a-705b-4772-8931-41eed21d7599 service cinder] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:47.092317 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9725931-7c60-4aff-8099-07d842451007 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:47.092978 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9725931-7c60-4aff-8099-07d842451007 service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:47.093254 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9725931-7c60-4aff-8099-07d842451007 service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:47.093528 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9725931-7c60-4aff-8099-07d842451007 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:47.107560 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9725931-7c60-4aff-8099-07d842451007 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:47.109441 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9725931-7c60-4aff-8099-07d842451007 service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:47.122927 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 817/1642] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:45:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:45:47.290503 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e9abde-ec8e-459f-8997-54d8eb081451 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:47.300297 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e9abde-ec8e-459f-8997-54d8eb081451 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:47.305426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8969499-37f3-499c-80b8-6ff4ad2488e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:47.305641 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8969499-37f3-499c-80b8-6ff4ad2488e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:47.305980 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8969499-37f3-499c-80b8-6ff4ad2488e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:47.306262 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8969499-37f3-499c-80b8-6ff4ad2488e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:47.318821 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8969499-37f3-499c-80b8-6ff4ad2488e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:47.320769 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8969499-37f3-499c-80b8-6ff4ad2488e7 service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:47.343103 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 826/1643] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:45:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:45:47.628076 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:45:47.762859 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9725931-7c60-4aff-8099-07d842451007 service cinder] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:47.773972 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9725931-7c60-4aff-8099-07d842451007 service cinder] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:47.779821 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce97b60-3e9b-40cd-a880-07059560a38e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:47.780533 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce97b60-3e9b-40cd-a880-07059560a38e service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:47.780821 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce97b60-3e9b-40cd-a880-07059560a38e service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:47.781586 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce97b60-3e9b-40cd-a880-07059560a38e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:47.861901 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce97b60-3e9b-40cd-a880-07059560a38e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:47.863297 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce97b60-3e9b-40cd-a880-07059560a38e service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:47.875173 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 818/1644] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:45:47 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:45:48.451200 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:45:48.475060 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8969499-37f3-499c-80b8-6ff4ad2488e7 service cinder] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:48.528800 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce97b60-3e9b-40cd-a880-07059560a38e service cinder] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:48.735128 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8969499-37f3-499c-80b8-6ff4ad2488e7 service cinder] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:48.753190 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec35c78-7955-40ce-9910-539b4786ee43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:48.754104 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec35c78-7955-40ce-9910-539b4786ee43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:48.754395 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec35c78-7955-40ce-9910-539b4786ee43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:48.754671 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec35c78-7955-40ce-9910-539b4786ee43 admin admin] PATH_INFO: `/v3/users/f2ddfbaed6c14218a89ea9d90e0b4ae8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:48.756237 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce97b60-3e9b-40cd-a880-07059560a38e service cinder] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:48.757350 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec35c78-7955-40ce-9910-539b4786ee43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2ddfbaed6c14218a89ea9d90e0b4ae8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:48.758345 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: WARNING py.warnings [None req-1ec35c78-7955-40ce-9910-539b4786ee43 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 16:45:48.758345 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: warnings.warn(msg) Jul 18 16:45:48.758345 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: Jul 18 16:45:48.759737 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec35c78-7955-40ce-9910-539b4786ee43 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:48.775039 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e45f86-3010-4718-86da-47b6eb5c7cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:48.776038 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e45f86-3010-4718-86da-47b6eb5c7cb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:48.776319 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e45f86-3010-4718-86da-47b6eb5c7cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:48.780223 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e45f86-3010-4718-86da-47b6eb5c7cb6 admin admin] PATH_INFO: `/v3/users/0a36d1f2e0b1457b81946ed9af4c05b0` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:48.787753 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e45f86-3010-4718-86da-47b6eb5c7cb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a36d1f2e0b1457b81946ed9af4c05b0)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:48.795330 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e45f86-3010-4718-86da-47b6eb5c7cb6 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:49.323075 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-22e45f86-3010-4718-86da-47b6eb5c7cb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a36d1f2e0b1457b81946ed9af4c05b0'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:45:49.331337 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-1ec35c78-7955-40ce-9910-539b4786ee43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2ddfbaed6c14218a89ea9d90e0b4ae8'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:45:49.424462 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-1ec35c78-7955-40ce-9910-539b4786ee43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2ddfbaed6c14218a89ea9d90e0b4ae8'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:45:49.437429 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-1ec35c78-7955-40ce-9910-539b4786ee43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2ddfbaed6c14218a89ea9d90e0b4ae8'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:45:49.478902 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-1ec35c78-7955-40ce-9910-539b4786ee43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2ddfbaed6c14218a89ea9d90e0b4ae8'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:45:49.479887 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-1ec35c78-7955-40ce-9910-539b4786ee43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2ddfbaed6c14218a89ea9d90e0b4ae8'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:45:49.488761 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 827/1645] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:45:48 2022] DELETE /identity/v3/users/f2ddfbaed6c14218a89ea9d90e0b4ae8 => generated 0 bytes in 1014 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:45:49.490276 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590c9db-3acd-44d5-85e5-0c1e87271dd4 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:49.490788 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590c9db-3acd-44d5-85e5-0c1e87271dd4 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:49.491348 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590c9db-3acd-44d5-85e5-0c1e87271dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:49.544694 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-f590c9db-3acd-44d5-85e5-0c1e87271dd4 None None] MFA Rules not processed for user `7852764385e34367936225432d9846cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:49.547898 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f590c9db-3acd-44d5-85e5-0c1e87271dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:49.573529 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-22e45f86-3010-4718-86da-47b6eb5c7cb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a36d1f2e0b1457b81946ed9af4c05b0'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:45:49.574766 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-22e45f86-3010-4718-86da-47b6eb5c7cb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a36d1f2e0b1457b81946ed9af4c05b0'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:45:49.575627 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-22e45f86-3010-4718-86da-47b6eb5c7cb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a36d1f2e0b1457b81946ed9af4c05b0'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:45:49.576477 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:45:49.592615 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-22e45f86-3010-4718-86da-47b6eb5c7cb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a36d1f2e0b1457b81946ed9af4c05b0'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:45:49.650031 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 819/1646] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:45:48 2022] DELETE /identity/v3/users/0a36d1f2e0b1457b81946ed9af4c05b0 => generated 0 bytes in 1121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:45:49.652013 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef066a-18d9-497a-93b1-aa61e75c2cb5 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:49.652504 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef066a-18d9-497a-93b1-aa61e75c2cb5 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:49.652953 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef066a-18d9-497a-93b1-aa61e75c2cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:49.678601 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 828/1647] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:45:49 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:45:49.679646 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-31ef066a-18d9-497a-93b1-aa61e75c2cb5 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:49.681133 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29896e62-0d66-4fdf-84fb-026c95cf413c None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:49.681431 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29896e62-0d66-4fdf-84fb-026c95cf413c None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:49.681712 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29896e62-0d66-4fdf-84fb-026c95cf413c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:49.683239 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-31ef066a-18d9-497a-93b1-aa61e75c2cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:49.707286 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-29896e62-0d66-4fdf-84fb-026c95cf413c None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:49.708857 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-29896e62-0d66-4fdf-84fb-026c95cf413c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:49.792447 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 829/1648] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:45:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:45:49.793907 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29896e62-0d66-4fdf-84fb-026c95cf413c None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:49.805210 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-29896e62-0d66-4fdf-84fb-026c95cf413c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:49.826778 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29896e62-0d66-4fdf-84fb-026c95cf413c None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:49.843033 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99be499-215c-4bee-a67d-56783739d210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:49.844696 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99be499-215c-4bee-a67d-56783739d210 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:49.846641 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99be499-215c-4bee-a67d-56783739d210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:49.848032 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99be499-215c-4bee-a67d-56783739d210 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:49.849541 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99be499-215c-4bee-a67d-56783739d210 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:49.853172 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99be499-215c-4bee-a67d-56783739d210 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:49.875837 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 820/1649] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:45:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:45:49.876464 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef066a-18d9-497a-93b1-aa61e75c2cb5 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:49.880163 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-31ef066a-18d9-497a-93b1-aa61e75c2cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:49.995051 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef066a-18d9-497a-93b1-aa61e75c2cb5 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:49.999815 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e77599-e262-4bed-acee-35072907efc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:50.000797 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e77599-e262-4bed-acee-35072907efc2 service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:50.001295 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e77599-e262-4bed-acee-35072907efc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:50.001781 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e77599-e262-4bed-acee-35072907efc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:50.019827 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e77599-e262-4bed-acee-35072907efc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:50.021735 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e77599-e262-4bed-acee-35072907efc2 service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:50.045311 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 821/1650] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:45:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:45:50.046413 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e77599-e262-4bed-acee-35072907efc2 service neutron] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:50.055729 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-86e77599-e262-4bed-acee-35072907efc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:50.073567 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e77599-e262-4bed-acee-35072907efc2 service neutron] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:50.079929 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295dc34c-5408-4329-9f4d-27ec4f161c8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:50.080710 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295dc34c-5408-4329-9f4d-27ec4f161c8b service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:50.081118 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295dc34c-5408-4329-9f4d-27ec4f161c8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:50.081501 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295dc34c-5408-4329-9f4d-27ec4f161c8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:50.092683 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295dc34c-5408-4329-9f4d-27ec4f161c8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:50.094528 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295dc34c-5408-4329-9f4d-27ec4f161c8b service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:50.110457 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 822/1651] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:45:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:45:50.576376 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9e63d1-a7e4-4615-8329-a20d182b5f5f None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:50.577020 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9e63d1-a7e4-4615-8329-a20d182b5f5f None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:50.577020 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9e63d1-a7e4-4615-8329-a20d182b5f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:50.583009 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 830/1652] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:45:49 2022] POST /identity/v3/projects => generated 405 bytes in 787 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:45:50.585026 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99be499-215c-4bee-a67d-56783739d210 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:50.608809 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-bc9e63d1-a7e4-4615-8329-a20d182b5f5f None None] MFA Rules not processed for user `87094997b3b94ebea3de60446795fb7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:50.610357 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-bc9e63d1-a7e4-4615-8329-a20d182b5f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:50.702651 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99be499-215c-4bee-a67d-56783739d210 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:50.717514 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e01c32a-5741-442a-8cdb-0047e9ba3946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:50.718276 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e01c32a-5741-442a-8cdb-0047e9ba3946 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:50.718788 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e01c32a-5741-442a-8cdb-0047e9ba3946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:50.718900 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e01c32a-5741-442a-8cdb-0047e9ba3946 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:50.719969 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e01c32a-5741-442a-8cdb-0047e9ba3946 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:50.721563 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e01c32a-5741-442a-8cdb-0047e9ba3946 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:50.749912 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 823/1653] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:45:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:45:50.752258 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488b4b9-2a41-4336-a97b-c6108414887e None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:50.752553 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488b4b9-2a41-4336-a97b-c6108414887e None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:50.755074 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488b4b9-2a41-4336-a97b-c6108414887e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:50.792518 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-3488b4b9-2a41-4336-a97b-c6108414887e None None] MFA Rules not processed for user `87094997b3b94ebea3de60446795fb7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:50.795105 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-3488b4b9-2a41-4336-a97b-c6108414887e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:50.810122 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 824/1654] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:45:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:45:50.827761 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3488b4b9-2a41-4336-a97b-c6108414887e None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:50.829467 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-3488b4b9-2a41-4336-a97b-c6108414887e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:50.863370 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:45:50.867649 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 831/1655] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:45:50 2022] POST /identity/v3/users => generated 309 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:45:50.872174 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e01c32a-5741-442a-8cdb-0047e9ba3946 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:50.895811 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e01c32a-5741-442a-8cdb-0047e9ba3946 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:50.911448 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c227c9a6-82eb-4258-8ba9-e5ffd4f3a1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:50.912721 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c227c9a6-82eb-4258-8ba9-e5ffd4f3a1ba admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:50.913210 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c227c9a6-82eb-4258-8ba9-e5ffd4f3a1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:50.913699 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c227c9a6-82eb-4258-8ba9-e5ffd4f3a1ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:50.916437 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c227c9a6-82eb-4258-8ba9-e5ffd4f3a1ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:50.918327 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c227c9a6-82eb-4258-8ba9-e5ffd4f3a1ba admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:50.953864 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 832/1656] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:45:50 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:45:50.955980 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c227c9a6-82eb-4258-8ba9-e5ffd4f3a1ba admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:50.979212 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c227c9a6-82eb-4258-8ba9-e5ffd4f3a1ba admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:50.984853 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50714b40-27ba-4572-a34e-2fd8d7de7672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:50.986077 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50714b40-27ba-4572-a34e-2fd8d7de7672 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:50.986518 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50714b40-27ba-4572-a34e-2fd8d7de7672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:50.987122 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50714b40-27ba-4572-a34e-2fd8d7de7672 admin admin] PATH_INFO: `/v3/projects/b8766e84cf8d43e0b060e98a39fcf594/users/8db76efa553f4e50ad606dc8713f41b3/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:51.020206 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3488b4b9-2a41-4336-a97b-c6108414887e None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:51.028546 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50714b40-27ba-4572-a34e-2fd8d7de7672 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8766e84cf8d43e0b060e98a39fcf594, user_id=8db76efa553f4e50ad606dc8713f41b3, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:51.038212 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50714b40-27ba-4572-a34e-2fd8d7de7672 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:51.044227 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d9c87-97a4-4311-af87-07e5b179761f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:51.047593 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d9c87-97a4-4311-af87-07e5b179761f service nova] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:51.047930 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d9c87-97a4-4311-af87-07e5b179761f service nova] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:51.048368 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d9c87-97a4-4311-af87-07e5b179761f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:51.089819 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328d9c87-97a4-4311-af87-07e5b179761f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:51.092603 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328d9c87-97a4-4311-af87-07e5b179761f service nova] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:51.119691 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 825/1657] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:45:50 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:45:51.156597 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 833/1658] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:45:50 2022] PUT /identity/v3/projects/b8766e84cf8d43e0b060e98a39fcf594/users/8db76efa553f4e50ad606dc8713f41b3/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:45:51.166070 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d9c87-97a4-4311-af87-07e5b179761f service nova] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:51.197681 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d9c87-97a4-4311-af87-07e5b179761f service nova] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:51.246768 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50714b40-27ba-4572-a34e-2fd8d7de7672 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:51.253499 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-50714b40-27ba-4572-a34e-2fd8d7de7672 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:51.339836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80960f22-47ba-4ac4-88cc-37234c0f9570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:51.340606 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80960f22-47ba-4ac4-88cc-37234c0f9570 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:51.340892 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80960f22-47ba-4ac4-88cc-37234c0f9570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:51.341174 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80960f22-47ba-4ac4-88cc-37234c0f9570 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:51.341917 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80960f22-47ba-4ac4-88cc-37234c0f9570 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:51.343954 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80960f22-47ba-4ac4-88cc-37234c0f9570 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:51.395009 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 826/1659] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:45:51 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:45:51.404251 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80960f22-47ba-4ac4-88cc-37234c0f9570 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:51.408283 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50714b40-27ba-4572-a34e-2fd8d7de7672 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:51.423858 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea49ff2a-8097-4eec-8d18-750102d1b313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:51.424574 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49ff2a-8097-4eec-8d18-750102d1b313 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:51.432059 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49ff2a-8097-4eec-8d18-750102d1b313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:51.432059 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49ff2a-8097-4eec-8d18-750102d1b313 admin admin] PATH_INFO: `/v3/projects/47f8e892e9284e2f926d1a23b80f0a5c` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:51.434220 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80960f22-47ba-4ac4-88cc-37234c0f9570 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:51.434982 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea49ff2a-8097-4eec-8d18-750102d1b313 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47f8e892e9284e2f926d1a23b80f0a5c)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:51.436968 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea49ff2a-8097-4eec-8d18-750102d1b313 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:51.439514 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a62206-4adb-4672-8609-10fe068cb935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:51.440448 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a62206-4adb-4672-8609-10fe068cb935 admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:51.440730 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a62206-4adb-4672-8609-10fe068cb935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:51.441077 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a62206-4adb-4672-8609-10fe068cb935 admin admin] PATH_INFO: `/v3/projects/b8766e84cf8d43e0b060e98a39fcf594/users/8db76efa553f4e50ad606dc8713f41b3/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:51.443966 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a62206-4adb-4672-8609-10fe068cb935 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8766e84cf8d43e0b060e98a39fcf594, user_id=8db76efa553f4e50ad606dc8713f41b3, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:51.447158 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a62206-4adb-4672-8609-10fe068cb935 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:51.461989 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 827/1660] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:45:51 2022] PUT /identity/v3/projects/b8766e84cf8d43e0b060e98a39fcf594/users/8db76efa553f4e50ad606dc8713f41b3/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:45:51.463077 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a62206-4adb-4672-8609-10fe068cb935 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:51.465199 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-44a62206-4adb-4672-8609-10fe068cb935 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:51.587447 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a62206-4adb-4672-8609-10fe068cb935 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:51.592900 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68439116-f62b-45a5-9b53-40f57a461a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:51.595274 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68439116-f62b-45a5-9b53-40f57a461a54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:51.595617 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68439116-f62b-45a5-9b53-40f57a461a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:51.595895 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68439116-f62b-45a5-9b53-40f57a461a54 admin admin] PATH_INFO: `/v3/projects/4bcee7e01003431db156174be70a8577` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:51.597250 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68439116-f62b-45a5-9b53-40f57a461a54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bcee7e01003431db156174be70a8577)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:51.598608 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: WARNING py.warnings [None req-68439116-f62b-45a5-9b53-40f57a461a54 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 16:45:51.598608 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: warnings.warn(msg) Jul 18 16:45:51.598608 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: Jul 18 16:45:51.600220 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68439116-f62b-45a5-9b53-40f57a461a54 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:51.625345 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-ea49ff2a-8097-4eec-8d18-750102d1b313 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47f8e892e9284e2f926d1a23b80f0a5c', 'request_id': 'req-ea49ff2a-8097-4eec-8d18-750102d1b313'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:45:51.664500 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-ea49ff2a-8097-4eec-8d18-750102d1b313 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47f8e892e9284e2f926d1a23b80f0a5c', 'request_id': 'req-ea49ff2a-8097-4eec-8d18-750102d1b313'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:45:51.676010 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-ea49ff2a-8097-4eec-8d18-750102d1b313 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47f8e892e9284e2f926d1a23b80f0a5c', 'request_id': 'req-ea49ff2a-8097-4eec-8d18-750102d1b313'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:45:51.828596 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-68439116-f62b-45a5-9b53-40f57a461a54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bcee7e01003431db156174be70a8577', 'request_id': 'req-68439116-f62b-45a5-9b53-40f57a461a54'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:45:51.846993 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-68439116-f62b-45a5-9b53-40f57a461a54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bcee7e01003431db156174be70a8577', 'request_id': 'req-68439116-f62b-45a5-9b53-40f57a461a54'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:45:51.863242 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-ea49ff2a-8097-4eec-8d18-750102d1b313 admin admin] The token cache is being invalidate because project 47f8e892e9284e2f926d1a23b80f0a5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:45:51.866221 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-ea49ff2a-8097-4eec-8d18-750102d1b313 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:45:51.866221 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-ea49ff2a-8097-4eec-8d18-750102d1b313 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:45:51.867822 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 834/1661] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:45:51 2022] DELETE /identity/v3/projects/47f8e892e9284e2f926d1a23b80f0a5c => generated 0 bytes in 623 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:45:51.873677 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32ed19-bbb4-4362-86e7-a499ff9f578a None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:51.874281 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32ed19-bbb4-4362-86e7-a499ff9f578a None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:51.874777 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32ed19-bbb4-4362-86e7-a499ff9f578a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:51.899154 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-bc32ed19-bbb4-4362-86e7-a499ff9f578a None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:51.899620 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-bc32ed19-bbb4-4362-86e7-a499ff9f578a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:51.963083 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-68439116-f62b-45a5-9b53-40f57a461a54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bcee7e01003431db156174be70a8577', 'request_id': 'req-68439116-f62b-45a5-9b53-40f57a461a54'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:45:51.988332 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 835/1662] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:45:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:45:51.996471 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe33be-ee5b-42b5-bb17-0faf7d85c609 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:51.996471 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe33be-ee5b-42b5-bb17-0faf7d85c609 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:51.996830 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe33be-ee5b-42b5-bb17-0faf7d85c609 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:52.008156 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 836/1663] 10.222.0.26 () {54 vars in 912 bytes} [Mon Jul 18 16:45:51 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:45:52.008156 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f411dc1a-a216-497a-89e8-4b09775fd212 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:52.008156 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f411dc1a-a216-497a-89e8-4b09775fd212 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:52.008156 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f411dc1a-a216-497a-89e8-4b09775fd212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:52.010416 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:INSERT=2,keystone:DELETE=3,keystone:SELECT=95 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:45:52.013130 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-68439116-f62b-45a5-9b53-40f57a461a54 admin admin] The token cache is being invalidate because project 4bcee7e01003431db156174be70a8577 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:45:52.013726 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-68439116-f62b-45a5-9b53-40f57a461a54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:45:52.015592 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-68439116-f62b-45a5-9b53-40f57a461a54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:45:52.018616 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 828/1664] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:45:51 2022] DELETE /identity/v3/projects/4bcee7e01003431db156174be70a8577 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:45:52.024753 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117979c6-ae84-4fda-b1e6-50ea59caa9d3 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:52.027105 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117979c6-ae84-4fda-b1e6-50ea59caa9d3 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:52.028853 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117979c6-ae84-4fda-b1e6-50ea59caa9d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:52.066798 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-f411dc1a-a216-497a-89e8-4b09775fd212 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:52.066798 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f411dc1a-a216-497a-89e8-4b09775fd212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:52.088254 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 837/1665] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:45:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:45:52.089272 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f411dc1a-a216-497a-89e8-4b09775fd212 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:52.090722 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f411dc1a-a216-497a-89e8-4b09775fd212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:52.121373 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f411dc1a-a216-497a-89e8-4b09775fd212 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:52.128040 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6998b424-1bbb-46bb-8250-b5a951004ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:52.128994 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6998b424-1bbb-46bb-8250-b5a951004ae2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:52.129280 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6998b424-1bbb-46bb-8250-b5a951004ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:52.129570 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6998b424-1bbb-46bb-8250-b5a951004ae2 admin admin] PATH_INFO: `/v3/users/5cae474a74e44b08a3b62d1877e0e40d` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:52.130809 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6998b424-1bbb-46bb-8250-b5a951004ae2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cae474a74e44b08a3b62d1877e0e40d)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:52.135221 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6998b424-1bbb-46bb-8250-b5a951004ae2 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:52.195826 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-117979c6-ae84-4fda-b1e6-50ea59caa9d3 None None] MFA Rules not processed for user `7a6054dbfbbc487794d2a32061b0ac82`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:52.197544 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-117979c6-ae84-4fda-b1e6-50ea59caa9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:52.349139 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 829/1666] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jul 18 16:45:52 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:45:52.351791 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117979c6-ae84-4fda-b1e6-50ea59caa9d3 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:52.387707 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117979c6-ae84-4fda-b1e6-50ea59caa9d3 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:52.420528 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6199fd25-ced8-4cad-abff-6117248df43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:52.421779 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199fd25-ced8-4cad-abff-6117248df43e admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:52.426282 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199fd25-ced8-4cad-abff-6117248df43e admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:52.427186 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199fd25-ced8-4cad-abff-6117248df43e admin admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:52.434491 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6199fd25-ced8-4cad-abff-6117248df43e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:45:52.434491 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6199fd25-ced8-4cad-abff-6117248df43e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:52.443073 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6199fd25-ced8-4cad-abff-6117248df43e admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:52.501056 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 830/1667] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 16:45:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:45:52.506747 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-6998b424-1bbb-46bb-8250-b5a951004ae2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cae474a74e44b08a3b62d1877e0e40d'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:45:52.508664 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6199fd25-ced8-4cad-abff-6117248df43e admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:52.520156 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-6199fd25-ced8-4cad-abff-6117248df43e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:52.551929 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6199fd25-ced8-4cad-abff-6117248df43e admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:52.559143 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a0af7e-70bc-45ca-a364-2a426ac0d5c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:52.561016 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a0af7e-70bc-45ca-a364-2a426ac0d5c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:52.561503 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a0af7e-70bc-45ca-a364-2a426ac0d5c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:52.561794 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a0af7e-70bc-45ca-a364-2a426ac0d5c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:52.580887 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a0af7e-70bc-45ca-a364-2a426ac0d5c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:52.580887 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a0af7e-70bc-45ca-a364-2a426ac0d5c7 service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:52.604557 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 831/1668] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:45:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:45:52.607084 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-6998b424-1bbb-46bb-8250-b5a951004ae2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cae474a74e44b08a3b62d1877e0e40d'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:45:52.621058 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-6998b424-1bbb-46bb-8250-b5a951004ae2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cae474a74e44b08a3b62d1877e0e40d'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:45:52.621244 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a0af7e-70bc-45ca-a364-2a426ac0d5c7 service neutron] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:52.689513 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-6998b424-1bbb-46bb-8250-b5a951004ae2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cae474a74e44b08a3b62d1877e0e40d'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:45:52.690129 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a0af7e-70bc-45ca-a364-2a426ac0d5c7 service neutron] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:52.693228 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-6998b424-1bbb-46bb-8250-b5a951004ae2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cae474a74e44b08a3b62d1877e0e40d'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:45:52.707074 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a1516e-96c2-4e4f-a4a8-ab5fbb45c7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:52.708938 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a1516e-96c2-4e4f-a4a8-ab5fbb45c7f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:52.709205 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a1516e-96c2-4e4f-a4a8-ab5fbb45c7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:52.709481 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a1516e-96c2-4e4f-a4a8-ab5fbb45c7f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:52.710196 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a1516e-96c2-4e4f-a4a8-ab5fbb45c7f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:52.715014 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a1516e-96c2-4e4f-a4a8-ab5fbb45c7f8 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:52.719577 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 838/1669] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:45:52 2022] DELETE /identity/v3/users/5cae474a74e44b08a3b62d1877e0e40d => generated 0 bytes in 628 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:45:52.850485 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 832/1670] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:45:52 2022] POST /identity/v3/projects => generated 373 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:45:52.852520 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6998b424-1bbb-46bb-8250-b5a951004ae2 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:52.854244 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-6998b424-1bbb-46bb-8250-b5a951004ae2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:52.979346 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:45:52.986549 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6998b424-1bbb-46bb-8250-b5a951004ae2 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:53.109527 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2648850-6382-415f-b46b-5e1ab258e2d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:53.110242 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2648850-6382-415f-b46b-5e1ab258e2d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:53.110528 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2648850-6382-415f-b46b-5e1ab258e2d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:53.110814 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2648850-6382-415f-b46b-5e1ab258e2d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:53.111570 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2648850-6382-415f-b46b-5e1ab258e2d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:53.113617 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2648850-6382-415f-b46b-5e1ab258e2d0 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:53.259127 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 839/1671] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:45:52 2022] POST /identity/v3/users => generated 287 bytes in 408 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:45:53.264918 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a1516e-96c2-4e4f-a4a8-ab5fbb45c7f8 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:53.283948 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a1516e-96c2-4e4f-a4a8-ab5fbb45c7f8 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:53.298729 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6c3a2c-a28a-4d98-9625-cddb4125cc35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:53.299509 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6c3a2c-a28a-4d98-9625-cddb4125cc35 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:53.299851 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6c3a2c-a28a-4d98-9625-cddb4125cc35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:53.300134 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6c3a2c-a28a-4d98-9625-cddb4125cc35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:53.301026 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6c3a2c-a28a-4d98-9625-cddb4125cc35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:53.309614 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6c3a2c-a28a-4d98-9625-cddb4125cc35 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:53.338941 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 833/1672] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:45:53 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:45:53.340436 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2648850-6382-415f-b46b-5e1ab258e2d0 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:53.357936 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2648850-6382-415f-b46b-5e1ab258e2d0 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:53.388854 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01600894-8037-47ba-a073-05204d80ad9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:53.394669 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01600894-8037-47ba-a073-05204d80ad9b admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:53.394977 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01600894-8037-47ba-a073-05204d80ad9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:53.397612 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01600894-8037-47ba-a073-05204d80ad9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:53.405511 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01600894-8037-47ba-a073-05204d80ad9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:53.407920 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01600894-8037-47ba-a073-05204d80ad9b admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:53.500715 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6c3a2c-a28a-4d98-9625-cddb4125cc35 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:53.506273 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 840/1673] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:45:53 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:45:53.524132 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6c3a2c-a28a-4d98-9625-cddb4125cc35 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:53.550701 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d34915-5a4f-495a-a906-2e323df821ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:53.551892 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d34915-5a4f-495a-a906-2e323df821ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:53.552375 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d34915-5a4f-495a-a906-2e323df821ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:53.552749 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d34915-5a4f-495a-a906-2e323df821ab admin admin] PATH_INFO: `/v3/projects/536017dde63546ff99f8cfef2890dc5a/users/9d219e2dd3a14573b67858bda9eb346b/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:53.596912 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d34915-5a4f-495a-a906-2e323df821ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=536017dde63546ff99f8cfef2890dc5a, user_id=9d219e2dd3a14573b67858bda9eb346b, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:53.596912 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d34915-5a4f-495a-a906-2e323df821ab admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:53.684163 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01600894-8037-47ba-a073-05204d80ad9b admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:53.686168 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-01600894-8037-47ba-a073-05204d80ad9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:53.723417 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 834/1674] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:45:53 2022] PUT /identity/v3/projects/536017dde63546ff99f8cfef2890dc5a/users/9d219e2dd3a14573b67858bda9eb346b/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:45:53.725215 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb1c3d-5694-4e9d-a0df-470f98366185 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:53.730779 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb1c3d-5694-4e9d-a0df-470f98366185 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:53.731160 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb1c3d-5694-4e9d-a0df-470f98366185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:53.777062 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01600894-8037-47ba-a073-05204d80ad9b admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:53.858623 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-f6eb1c3d-5694-4e9d-a0df-470f98366185 None None] MFA Rules not processed for user `9d219e2dd3a14573b67858bda9eb346b`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:53.864637 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-f6eb1c3d-5694-4e9d-a0df-470f98366185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:54.017217 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493aa803-2d86-46e4-82cb-5e7430252c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:54.018103 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493aa803-2d86-46e4-82cb-5e7430252c1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:54.018388 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493aa803-2d86-46e4-82cb-5e7430252c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:54.018694 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493aa803-2d86-46e4-82cb-5e7430252c1d admin admin] PATH_INFO: `/v3/projects/1931dfb579c54941b9ea02300866be98` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:54.025919 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493aa803-2d86-46e4-82cb-5e7430252c1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1931dfb579c54941b9ea02300866be98)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:54.029212 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493aa803-2d86-46e4-82cb-5e7430252c1d admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:54.099788 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 835/1675] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:45:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:45:54.101732 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c779483-8651-4837-bf30-162acfaf480e None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:54.102024 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c779483-8651-4837-bf30-162acfaf480e None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:54.103804 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c779483-8651-4837-bf30-162acfaf480e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:54.171840 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-4c779483-8651-4837-bf30-162acfaf480e None None] MFA Rules not processed for user `8db76efa553f4e50ad606dc8713f41b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:54.173616 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-4c779483-8651-4837-bf30-162acfaf480e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:54.307592 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-493aa803-2d86-46e4-82cb-5e7430252c1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1931dfb579c54941b9ea02300866be98', 'request_id': 'req-493aa803-2d86-46e4-82cb-5e7430252c1d'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:45:54.316140 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-493aa803-2d86-46e4-82cb-5e7430252c1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1931dfb579c54941b9ea02300866be98', 'request_id': 'req-493aa803-2d86-46e4-82cb-5e7430252c1d'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:45:54.332909 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-493aa803-2d86-46e4-82cb-5e7430252c1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1931dfb579c54941b9ea02300866be98', 'request_id': 'req-493aa803-2d86-46e4-82cb-5e7430252c1d'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:45:54.366015 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 836/1676] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:45:54 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:45:54.367525 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c779483-8651-4837-bf30-162acfaf480e None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:54.382925 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c779483-8651-4837-bf30-162acfaf480e None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:54.446014 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-493aa803-2d86-46e4-82cb-5e7430252c1d admin admin] The token cache is being invalidate because project 1931dfb579c54941b9ea02300866be98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:45:54.446578 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-493aa803-2d86-46e4-82cb-5e7430252c1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:45:54.447471 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-493aa803-2d86-46e4-82cb-5e7430252c1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:45:54.449692 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 841/1677] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:45:53 2022] DELETE /identity/v3/projects/1931dfb579c54941b9ea02300866be98 => generated 0 bytes in 768 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:45:54.455579 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1924f52-64e4-47f1-b8e0-1d579186772f None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:54.456160 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1924f52-64e4-47f1-b8e0-1d579186772f None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:54.457309 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1924f52-64e4-47f1-b8e0-1d579186772f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:54.485542 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-d1924f52-64e4-47f1-b8e0-1d579186772f None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:54.487107 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d1924f52-64e4-47f1-b8e0-1d579186772f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:54.497780 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af87092b-a3b2-4e15-b48a-33845cce847f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:54.498841 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af87092b-a3b2-4e15-b48a-33845cce847f admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:54.499981 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af87092b-a3b2-4e15-b48a-33845cce847f admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:54.500539 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af87092b-a3b2-4e15-b48a-33845cce847f admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:54.501534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af87092b-a3b2-4e15-b48a-33845cce847f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:54.508406 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af87092b-a3b2-4e15-b48a-33845cce847f admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:54.595007 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 842/1678] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:45:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:45:54.596887 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07b82d-e681-47a8-8207-1ff3a8e214af None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:54.597355 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07b82d-e681-47a8-8207-1ff3a8e214af None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:54.597784 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07b82d-e681-47a8-8207-1ff3a8e214af None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:54.599865 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 843/1679] 10.222.0.26 () {54 vars in 961 bytes} [Mon Jul 18 16:45:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:45:54.604269 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1323b9e-768c-4660-bd3a-6c3cec1319c3 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:54.604749 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1323b9e-768c-4660-bd3a-6c3cec1319c3 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:54.605365 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1323b9e-768c-4660-bd3a-6c3cec1319c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:54.622316 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 837/1680] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:45:54 2022] POST /identity/v3/projects => generated 371 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:45:54.623918 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9331857-846f-42a9-bd58-d0c483acc363 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:54.625253 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9331857-846f-42a9-bd58-d0c483acc363 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:54.625550 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9331857-846f-42a9-bd58-d0c483acc363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:54.637155 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-b1323b9e-768c-4660-bd3a-6c3cec1319c3 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:54.638845 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-b1323b9e-768c-4660-bd3a-6c3cec1319c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:54.667921 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-d9331857-846f-42a9-bd58-d0c483acc363 None None] MFA Rules not processed for user `70f704a231a5474e8067a6c2dd88b355`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:54.671990 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-d9331857-846f-42a9-bd58-d0c483acc363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:54.720380 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:45:54.796690 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 838/1681] 10.222.0.26 () {58 vars in 1058 bytes} [Mon Jul 18 16:45:54 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:45:54.797608 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9331857-846f-42a9-bd58-d0c483acc363 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:54.798780 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 844/1682] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:45:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:45:54.799300 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-d9331857-846f-42a9-bd58-d0c483acc363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:54.802783 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1323b9e-768c-4660-bd3a-6c3cec1319c3 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:54.818202 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9331857-846f-42a9-bd58-d0c483acc363 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:54.821143 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1323b9e-768c-4660-bd3a-6c3cec1319c3 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:54.826184 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ad3ba2-43ab-4501-aa70-b22926f78f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:54.827175 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad3ba2-43ab-4501-aa70-b22926f78f44 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:54.827704 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad3ba2-43ab-4501-aa70-b22926f78f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:54.828200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad3ba2-43ab-4501-aa70-b22926f78f44 admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:54.829139 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ad3ba2-43ab-4501-aa70-b22926f78f44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:45:54.829642 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ad3ba2-43ab-4501-aa70-b22926f78f44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:54.832624 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ad3ba2-43ab-4501-aa70-b22926f78f44 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:54.833417 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8443e4-54c2-4f04-8e29-530395de2f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:54.834170 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8443e4-54c2-4f04-8e29-530395de2f09 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:54.834459 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8443e4-54c2-4f04-8e29-530395de2f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:54.834854 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8443e4-54c2-4f04-8e29-530395de2f09 admin admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:54.836403 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8443e4-54c2-4f04-8e29-530395de2f09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:54.839706 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8443e4-54c2-4f04-8e29-530395de2f09 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:54.848454 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 845/1683] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 16:45:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:45:54.849609 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ad3ba2-43ab-4501-aa70-b22926f78f44 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:54.864512 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ad3ba2-43ab-4501-aa70-b22926f78f44 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:54.869706 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e041f590-562f-43e2-bea8-bf7c6751ddfa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70f704a231a5474e8067a6c2dd88b355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:54.870361 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e041f590-562f-43e2-bea8-bf7c6751ddfa service placement] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:54.870645 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e041f590-562f-43e2-bea8-bf7c6751ddfa service placement] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:54.870922 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e041f590-562f-43e2-bea8-bf7c6751ddfa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:54.872797 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e041f590-562f-43e2-bea8-bf7c6751ddfa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:54.886457 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e041f590-562f-43e2-bea8-bf7c6751ddfa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:54.888608 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e041f590-562f-43e2-bea8-bf7c6751ddfa service placement] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:54.902331 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 846/1684] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:45:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:45:54.912610 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e041f590-562f-43e2-bea8-bf7c6751ddfa service placement] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:54.929945 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e041f590-562f-43e2-bea8-bf7c6751ddfa service placement] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:54.937312 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500162d4-1796-419f-8cbc-407cc3d4ee93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:54.938103 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500162d4-1796-419f-8cbc-407cc3d4ee93 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:54.938380 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500162d4-1796-419f-8cbc-407cc3d4ee93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:54.938704 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500162d4-1796-419f-8cbc-407cc3d4ee93 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:54.939990 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500162d4-1796-419f-8cbc-407cc3d4ee93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:54.942328 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500162d4-1796-419f-8cbc-407cc3d4ee93 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:54.945803 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 839/1685] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:45:54 2022] POST /identity/v3/users => generated 292 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:45:54.947161 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8443e4-54c2-4f04-8e29-530395de2f09 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:54.974332 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8443e4-54c2-4f04-8e29-530395de2f09 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:54.990260 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749dde2b-1798-4e9f-a864-b4ad700ca9af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:54.991136 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749dde2b-1798-4e9f-a864-b4ad700ca9af admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:54.991415 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749dde2b-1798-4e9f-a864-b4ad700ca9af admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:54.991886 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749dde2b-1798-4e9f-a864-b4ad700ca9af admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:54.992808 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749dde2b-1798-4e9f-a864-b4ad700ca9af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:54.994932 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749dde2b-1798-4e9f-a864-b4ad700ca9af admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:55.021704 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 840/1686] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:45:54 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:45:55.026651 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749dde2b-1798-4e9f-a864-b4ad700ca9af admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:55.061876 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:45:55.065937 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 847/1687] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:45:54 2022] POST /identity/v3/projects => generated 379 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:45:55.069511 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500162d4-1796-419f-8cbc-407cc3d4ee93 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:55.075082 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749dde2b-1798-4e9f-a864-b4ad700ca9af admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:55.229129 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c381859-06f6-4bcc-9bf7-147adfce913c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:55.230285 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c381859-06f6-4bcc-9bf7-147adfce913c admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:55.230714 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c381859-06f6-4bcc-9bf7-147adfce913c admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:55.231155 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c381859-06f6-4bcc-9bf7-147adfce913c admin admin] PATH_INFO: `/v3/projects/2dbe16bd89994b56a632e6f69f49e001/users/64ef62a6160e40c38c47bd91464e4a79/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:55.235536 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500162d4-1796-419f-8cbc-407cc3d4ee93 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:55.243166 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de555618-4220-4f60-88f3-ef6f12ed94bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:55.243952 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de555618-4220-4f60-88f3-ef6f12ed94bf admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:55.244242 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de555618-4220-4f60-88f3-ef6f12ed94bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:55.244524 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de555618-4220-4f60-88f3-ef6f12ed94bf admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:55.245308 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de555618-4220-4f60-88f3-ef6f12ed94bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:55.249640 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de555618-4220-4f60-88f3-ef6f12ed94bf admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:55.276141 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c381859-06f6-4bcc-9bf7-147adfce913c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dbe16bd89994b56a632e6f69f49e001, user_id=64ef62a6160e40c38c47bd91464e4a79, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:55.278874 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c381859-06f6-4bcc-9bf7-147adfce913c admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:55.384777 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 848/1688] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:45:55 2022] POST /identity/v3/users => generated 290 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:45:55.387410 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de555618-4220-4f60-88f3-ef6f12ed94bf admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:55.398817 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 841/1689] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:45:55 2022] PUT /identity/v3/projects/2dbe16bd89994b56a632e6f69f49e001/users/64ef62a6160e40c38c47bd91464e4a79/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:45:55.406786 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de555618-4220-4f60-88f3-ef6f12ed94bf admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:55.406982 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c381859-06f6-4bcc-9bf7-147adfce913c admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:55.496986 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0c50bc-353f-4baf-a2f5-626408b1e560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:55.499143 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c50bc-353f-4baf-a2f5-626408b1e560 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:55.499829 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c50bc-353f-4baf-a2f5-626408b1e560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:55.500305 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c50bc-353f-4baf-a2f5-626408b1e560 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:55.501331 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0c50bc-353f-4baf-a2f5-626408b1e560 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:55.501589 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c381859-06f6-4bcc-9bf7-147adfce913c admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:55.504443 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0c50bc-353f-4baf-a2f5-626408b1e560 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:55.506777 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d826731-3ff7-44be-a74b-3f922fc0458d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:55.507867 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d826731-3ff7-44be-a74b-3f922fc0458d admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:55.507977 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d826731-3ff7-44be-a74b-3f922fc0458d admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:55.508198 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d826731-3ff7-44be-a74b-3f922fc0458d admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:55.508929 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d826731-3ff7-44be-a74b-3f922fc0458d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:55.510742 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d826731-3ff7-44be-a74b-3f922fc0458d admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:55.532837 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 842/1690] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:45:55 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:45:55.536787 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d826731-3ff7-44be-a74b-3f922fc0458d admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:55.537596 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 849/1691] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:45:55 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:45:55.539186 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0c50bc-353f-4baf-a2f5-626408b1e560 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:55.556001 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0c50bc-353f-4baf-a2f5-626408b1e560 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:55.562236 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d826731-3ff7-44be-a74b-3f922fc0458d admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:55.562848 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f57954-6722-46fb-b6d2-20631d30f6f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:55.564224 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f57954-6722-46fb-b6d2-20631d30f6f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:55.564575 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f57954-6722-46fb-b6d2-20631d30f6f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:55.564975 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f57954-6722-46fb-b6d2-20631d30f6f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:55.565970 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f57954-6722-46fb-b6d2-20631d30f6f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:55.569009 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f57954-6722-46fb-b6d2-20631d30f6f7 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:55.575749 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cade80-6b7e-4390-bf6b-935b8b7c6b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:55.576728 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cade80-6b7e-4390-bf6b-935b8b7c6b00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:55.577012 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cade80-6b7e-4390-bf6b-935b8b7c6b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:55.577293 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cade80-6b7e-4390-bf6b-935b8b7c6b00 admin admin] PATH_INFO: `/v3/projects/2dbe16bd89994b56a632e6f69f49e001/users/64ef62a6160e40c38c47bd91464e4a79/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:55.579894 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cade80-6b7e-4390-bf6b-935b8b7c6b00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dbe16bd89994b56a632e6f69f49e001, user_id=64ef62a6160e40c38c47bd91464e4a79, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:55.584923 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cade80-6b7e-4390-bf6b-935b8b7c6b00 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:55.589060 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 850/1692] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:45:55 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:45:55.590876 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f57954-6722-46fb-b6d2-20631d30f6f7 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:55.599555 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 843/1693] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:45:55 2022] PUT /identity/v3/projects/2dbe16bd89994b56a632e6f69f49e001/users/64ef62a6160e40c38c47bd91464e4a79/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:45:55.605747 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f57954-6722-46fb-b6d2-20631d30f6f7 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:55.613668 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c254b58f-e77d-4686-96aa-461d7f06d809 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:55.614281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c254b58f-e77d-4686-96aa-461d7f06d809 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:55.614739 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c254b58f-e77d-4686-96aa-461d7f06d809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:55.675919 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-c254b58f-e77d-4686-96aa-461d7f06d809 None None] MFA Rules not processed for user `64ef62a6160e40c38c47bd91464e4a79`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:55.677443 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-c254b58f-e77d-4686-96aa-461d7f06d809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:55.778187 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3777c76-139e-4027-b33a-fb709634c0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:55.784030 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3777c76-139e-4027-b33a-fb709634c0da admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:55.784555 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3777c76-139e-4027-b33a-fb709634c0da admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:55.785253 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3777c76-139e-4027-b33a-fb709634c0da admin admin] PATH_INFO: `/v3/projects/5d368dc46b5147edbe03a809ed3be56b/users/9951616cfeee4a2ba8c77d147b5687f8/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:55.843988 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3777c76-139e-4027-b33a-fb709634c0da admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d368dc46b5147edbe03a809ed3be56b, user_id=9951616cfeee4a2ba8c77d147b5687f8, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:55.856491 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3777c76-139e-4027-b33a-fb709634c0da admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:55.885058 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 844/1694] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:45:55 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:45:55.902058 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c254b58f-e77d-4686-96aa-461d7f06d809 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:55.904067 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-c254b58f-e77d-4686-96aa-461d7f06d809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:55.957151 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 851/1695] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:45:55 2022] PUT /identity/v3/projects/5d368dc46b5147edbe03a809ed3be56b/users/9951616cfeee4a2ba8c77d147b5687f8/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:45:55.978786 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75df2cc-5e95-4952-93e8-3f2df0f1f5e8 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:55.979073 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75df2cc-5e95-4952-93e8-3f2df0f1f5e8 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:55.979339 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75df2cc-5e95-4952-93e8-3f2df0f1f5e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:56.029907 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-f75df2cc-5e95-4952-93e8-3f2df0f1f5e8 None None] MFA Rules not processed for user `9951616cfeee4a2ba8c77d147b5687f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:56.030673 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f75df2cc-5e95-4952-93e8-3f2df0f1f5e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:56.065177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c254b58f-e77d-4686-96aa-461d7f06d809 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:56.166556 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:45:56.172222 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e7edb2-4264-444c-9764-be649730d429 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:56.172924 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e7edb2-4264-444c-9764-be649730d429 service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:56.173280 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e7edb2-4264-444c-9764-be649730d429 service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:56.173574 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e7edb2-4264-444c-9764-be649730d429 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:56.190151 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 852/1696] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:45:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:45:56.201695 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d80a7-4a93-47fc-8d91-665b0d76c7a3 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:56.202628 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d80a7-4a93-47fc-8d91-665b0d76c7a3 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:56.203120 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d80a7-4a93-47fc-8d91-665b0d76c7a3 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:56.205248 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 853/1697] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jul 18 16:45:56 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:45:56.206337 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4d80a7-4a93-47fc-8d91-665b0d76c7a3 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:56.313419 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e7edb2-4264-444c-9764-be649730d429 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:56.314899 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e7edb2-4264-444c-9764-be649730d429 service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:56.317512 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:45:56.338379 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 845/1698] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:45:55 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:45:56.338806 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43189a1a-4f86-4a4e-99a3-4b25b66ea574 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:56.341472 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43189a1a-4f86-4a4e-99a3-4b25b66ea574 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:56.341822 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43189a1a-4f86-4a4e-99a3-4b25b66ea574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:56.357885 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4d80a7-4a93-47fc-8d91-665b0d76c7a3 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:56.373247 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6b5a07-cba5-4d92-86a8-de130caa44a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:56.374763 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b5a07-cba5-4d92-86a8-de130caa44a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:56.374763 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b5a07-cba5-4d92-86a8-de130caa44a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:56.374763 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b5a07-cba5-4d92-86a8-de130caa44a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:56.375201 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6b5a07-cba5-4d92-86a8-de130caa44a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:56.380400 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6b5a07-cba5-4d92-86a8-de130caa44a2 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:56.383543 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-43189a1a-4f86-4a4e-99a3-4b25b66ea574 None None] MFA Rules not processed for user `70f704a231a5474e8067a6c2dd88b355`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:56.390910 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-43189a1a-4f86-4a4e-99a3-4b25b66ea574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:56.535181 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 854/1699] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:45:56 2022] POST /identity/v3/projects => generated 379 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:45:56.540209 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6b5a07-cba5-4d92-86a8-de130caa44a2 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:56.658451 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 846/1700] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jul 18 16:45:56 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:45:56.740494 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43189a1a-4f86-4a4e-99a3-4b25b66ea574 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:56.762469 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43189a1a-4f86-4a4e-99a3-4b25b66ea574 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:56.765512 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6b5a07-cba5-4d92-86a8-de130caa44a2 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:56.807648 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f62fe-2e35-4441-b997-e234c7fd9054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:56.808369 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f62fe-2e35-4441-b997-e234c7fd9054 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:56.808644 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f62fe-2e35-4441-b997-e234c7fd9054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:56.808925 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f62fe-2e35-4441-b997-e234c7fd9054 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:56.809708 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456f62fe-2e35-4441-b997-e234c7fd9054 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:56.813914 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456f62fe-2e35-4441-b997-e234c7fd9054 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:56.949629 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 855/1701] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:45:56 2022] POST /identity/v3/users => generated 296 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:45:56.950573 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f62fe-2e35-4441-b997-e234c7fd9054 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:56.954058 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-456f62fe-2e35-4441-b997-e234c7fd9054 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:56.964648 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff748467-bd24-4783-acf8-683e7507c396 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:56.965335 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff748467-bd24-4783-acf8-683e7507c396 service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:56.965624 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff748467-bd24-4783-acf8-683e7507c396 service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:56.968168 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff748467-bd24-4783-acf8-683e7507c396 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:56.975361 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-ff748467-bd24-4783-acf8-683e7507c396 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:57.163285 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f62fe-2e35-4441-b997-e234c7fd9054 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:57.186549 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d36045a-748a-4ecf-9623-d90846cc013d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70f704a231a5474e8067a6c2dd88b355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:57.186549 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d36045a-748a-4ecf-9623-d90846cc013d service placement] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:57.186549 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d36045a-748a-4ecf-9623-d90846cc013d service placement] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:57.186549 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d36045a-748a-4ecf-9623-d90846cc013d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:57.186953 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff748467-bd24-4783-acf8-683e7507c396 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:57.189343 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff748467-bd24-4783-acf8-683e7507c396 service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:57.212597 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d36045a-748a-4ecf-9623-d90846cc013d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:57.219238 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d36045a-748a-4ecf-9623-d90846cc013d service placement] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:57.220677 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 847/1702] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:45:56 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:45:57.221869 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff748467-bd24-4783-acf8-683e7507c396 service cinder] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:57.242228 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff748467-bd24-4783-acf8-683e7507c396 service cinder] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:57.280066 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe0b7f-fab7-40f1-a023-fd7c5c434f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:57.280780 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe0b7f-fab7-40f1-a023-fd7c5c434f7f admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:57.281170 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe0b7f-fab7-40f1-a023-fd7c5c434f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:57.281334 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe0b7f-fab7-40f1-a023-fd7c5c434f7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:57.282192 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe0b7f-fab7-40f1-a023-fd7c5c434f7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:57.283836 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 856/1703] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:45:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:45:57.284860 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe0b7f-fab7-40f1-a023-fd7c5c434f7f admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:57.329545 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 848/1704] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:45:57 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:45:57.331247 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d36045a-748a-4ecf-9623-d90846cc013d service placement] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:57.346273 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d36045a-748a-4ecf-9623-d90846cc013d service placement] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:57.357720 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c66142-6aab-491a-b773-a87d9fc06080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:57.360768 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c66142-6aab-491a-b773-a87d9fc06080 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:57.361726 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c66142-6aab-491a-b773-a87d9fc06080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:57.362019 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c66142-6aab-491a-b773-a87d9fc06080 admin admin] PATH_INFO: `/v3/projects/e4cc2c112a8a44e892079f9169bdc3ae/users/d04f2a5c0a4d48af991e4c1c2df556ed/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:57.412470 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c66142-6aab-491a-b773-a87d9fc06080 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4cc2c112a8a44e892079f9169bdc3ae, user_id=d04f2a5c0a4d48af991e4c1c2df556ed, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:57.422102 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c66142-6aab-491a-b773-a87d9fc06080 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:57.603210 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 857/1705] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:45:57 2022] PUT /identity/v3/projects/e4cc2c112a8a44e892079f9169bdc3ae/users/d04f2a5c0a4d48af991e4c1c2df556ed/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:45:57.606247 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe0b7f-fab7-40f1-a023-fd7c5c434f7f admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:57.628208 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe0b7f-fab7-40f1-a023-fd7c5c434f7f admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:57.779718 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450f97c-4c75-4289-8315-639212287ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:57.780432 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450f97c-4c75-4289-8315-639212287ae4 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:57.780707 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450f97c-4c75-4289-8315-639212287ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:57.781063 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450f97c-4c75-4289-8315-639212287ae4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:57.781821 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e450f97c-4c75-4289-8315-639212287ae4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:57.783456 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e450f97c-4c75-4289-8315-639212287ae4 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:57.814044 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 849/1706] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:45:57 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:45:57.818422 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c66142-6aab-491a-b773-a87d9fc06080 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:57.836435 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c66142-6aab-491a-b773-a87d9fc06080 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:57.864212 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a609aebf-8bf6-4bde-b538-ad1f636893ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:57.869477 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609aebf-8bf6-4bde-b538-ad1f636893ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:57.869719 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609aebf-8bf6-4bde-b538-ad1f636893ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:57.874440 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609aebf-8bf6-4bde-b538-ad1f636893ee admin admin] PATH_INFO: `/v3/projects/e4cc2c112a8a44e892079f9169bdc3ae/users/d04f2a5c0a4d48af991e4c1c2df556ed/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:57.880553 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a609aebf-8bf6-4bde-b538-ad1f636893ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4cc2c112a8a44e892079f9169bdc3ae, user_id=d04f2a5c0a4d48af991e4c1c2df556ed, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:57.895718 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a609aebf-8bf6-4bde-b538-ad1f636893ee admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:57.941575 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 858/1707] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:45:57 2022] PUT /identity/v3/projects/e4cc2c112a8a44e892079f9169bdc3ae/users/d04f2a5c0a4d48af991e4c1c2df556ed/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:45:57.953043 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59635ad8-2edb-4208-b13e-4a13f512b0a3 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:57.953636 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59635ad8-2edb-4208-b13e-4a13f512b0a3 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:57.954105 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59635ad8-2edb-4208-b13e-4a13f512b0a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:57.992371 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-59635ad8-2edb-4208-b13e-4a13f512b0a3 None None] MFA Rules not processed for user `d04f2a5c0a4d48af991e4c1c2df556ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:57.995931 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-59635ad8-2edb-4208-b13e-4a13f512b0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:58.115254 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 850/1708] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:45:57 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:45:58.220183 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a609aebf-8bf6-4bde-b538-ad1f636893ee admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:58.379806 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a609aebf-8bf6-4bde-b538-ad1f636893ee admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:58.389071 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5ddb26-3b94-4e9c-9e83-d194cbbec8ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:58.390735 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ddb26-3b94-4e9c-9e83-d194cbbec8ad service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:58.391150 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ddb26-3b94-4e9c-9e83-d194cbbec8ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:58.392309 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ddb26-3b94-4e9c-9e83-d194cbbec8ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:58.439374 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59635ad8-2edb-4208-b13e-4a13f512b0a3 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:58.441065 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-59635ad8-2edb-4208-b13e-4a13f512b0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:58.469117 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5ddb26-3b94-4e9c-9e83-d194cbbec8ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:58.471995 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5ddb26-3b94-4e9c-9e83-d194cbbec8ad service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:58.524725 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 859/1709] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:45:58 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:45:58.672326 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59635ad8-2edb-4208-b13e-4a13f512b0a3 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:58.710070 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71601f9-2ecc-4b25-85da-20b0fee6819c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:58.710070 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71601f9-2ecc-4b25-85da-20b0fee6819c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:58.710070 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71601f9-2ecc-4b25-85da-20b0fee6819c admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:58.710070 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71601f9-2ecc-4b25-85da-20b0fee6819c admin admin] PATH_INFO: `/v3/users/7852764385e34367936225432d9846cd` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:58.711155 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71601f9-2ecc-4b25-85da-20b0fee6819c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7852764385e34367936225432d9846cd)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:58.711155 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71601f9-2ecc-4b25-85da-20b0fee6819c admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:58.842444 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:45:58.985941 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5ddb26-3b94-4e9c-9e83-d194cbbec8ad service cinder] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:58.987253 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-ec5ddb26-3b94-4e9c-9e83-d194cbbec8ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:59.003410 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5ddb26-3b94-4e9c-9e83-d194cbbec8ad service cinder] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:59.009039 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb5db2d-433d-42b6-ad74-b0d37fe99ab0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:59.009721 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5db2d-433d-42b6-ad74-b0d37fe99ab0 service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:59.010005 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5db2d-433d-42b6-ad74-b0d37fe99ab0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:59.010276 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5db2d-433d-42b6-ad74-b0d37fe99ab0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:59.100063 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:45:59.116311 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb5db2d-433d-42b6-ad74-b0d37fe99ab0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:59.118270 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb5db2d-433d-42b6-ad74-b0d37fe99ab0 service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:59.156183 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 860/1710] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:45:58 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:45:59.158569 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-a71601f9-2ecc-4b25-85da-20b0fee6819c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7852764385e34367936225432d9846cd'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:45:59.266595 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-a71601f9-2ecc-4b25-85da-20b0fee6819c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7852764385e34367936225432d9846cd'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:45:59.269176 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-a71601f9-2ecc-4b25-85da-20b0fee6819c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7852764385e34367936225432d9846cd'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:45:59.270803 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-a71601f9-2ecc-4b25-85da-20b0fee6819c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7852764385e34367936225432d9846cd'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:45:59.306455 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-a71601f9-2ecc-4b25-85da-20b0fee6819c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7852764385e34367936225432d9846cd'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:45:59.371423 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 851/1711] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:45:58 2022] DELETE /identity/v3/users/7852764385e34367936225432d9846cd => generated 0 bytes in 932 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:45:59.858644 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb5db2d-433d-42b6-ad74-b0d37fe99ab0 service cinder] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:59.876539 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb5db2d-433d-42b6-ad74-b0d37fe99ab0 service cinder] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:00.005509 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac31e7-bfe4-4410-8cd1-1db358031242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:00.006539 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac31e7-bfe4-4410-8cd1-1db358031242 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:00.006825 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac31e7-bfe4-4410-8cd1-1db358031242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:00.007100 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac31e7-bfe4-4410-8cd1-1db358031242 admin admin] PATH_INFO: `/v3/projects/2ca139607f5147ebb9acd9a03931f2f8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:00.014624 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ac31e7-bfe4-4410-8cd1-1db358031242 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ca139607f5147ebb9acd9a03931f2f8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:00.016968 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ac31e7-bfe4-4410-8cd1-1db358031242 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:00.171807 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-38ac31e7-bfe4-4410-8cd1-1db358031242 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ca139607f5147ebb9acd9a03931f2f8', 'request_id': 'req-38ac31e7-bfe4-4410-8cd1-1db358031242'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:00.194620 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-38ac31e7-bfe4-4410-8cd1-1db358031242 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ca139607f5147ebb9acd9a03931f2f8', 'request_id': 'req-38ac31e7-bfe4-4410-8cd1-1db358031242'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:00.228574 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-38ac31e7-bfe4-4410-8cd1-1db358031242 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ca139607f5147ebb9acd9a03931f2f8', 'request_id': 'req-38ac31e7-bfe4-4410-8cd1-1db358031242'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:00.376938 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-38ac31e7-bfe4-4410-8cd1-1db358031242 admin admin] The token cache is being invalidate because project 2ca139607f5147ebb9acd9a03931f2f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:46:00.379094 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-38ac31e7-bfe4-4410-8cd1-1db358031242 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:00.381722 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-38ac31e7-bfe4-4410-8cd1-1db358031242 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:00.384753 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 861/1712] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:45:59 2022] DELETE /identity/v3/projects/2ca139607f5147ebb9acd9a03931f2f8 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:00.388834 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71601f9-2ecc-4b25-85da-20b0fee6819c admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:00.390869 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-a71601f9-2ecc-4b25-85da-20b0fee6819c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:00.430511 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71601f9-2ecc-4b25-85da-20b0fee6819c admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:00.470753 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac31e7-bfe4-4410-8cd1-1db358031242 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:00.470753 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-38ac31e7-bfe4-4410-8cd1-1db358031242 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:00.605826 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64bfc68-c7ac-479d-99e2-c6f1f554ff58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:00.606783 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64bfc68-c7ac-479d-99e2-c6f1f554ff58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:00.616330 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64bfc68-c7ac-479d-99e2-c6f1f554ff58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:00.616667 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64bfc68-c7ac-479d-99e2-c6f1f554ff58 admin admin] PATH_INFO: `/v3/users/8db76efa553f4e50ad606dc8713f41b3` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:00.619427 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64bfc68-c7ac-479d-99e2-c6f1f554ff58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8db76efa553f4e50ad606dc8713f41b3)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:00.621439 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64bfc68-c7ac-479d-99e2-c6f1f554ff58 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:00.658339 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac31e7-bfe4-4410-8cd1-1db358031242 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:00.658614 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:00.686811 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ccb734-dcf7-428d-9e67-69bbf817ed6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:00.688125 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ccb734-dcf7-428d-9e67-69bbf817ed6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:00.688645 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ccb734-dcf7-428d-9e67-69bbf817ed6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:00.689238 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ccb734-dcf7-428d-9e67-69bbf817ed6e admin admin] PATH_INFO: `/v3/users/d6cfe76f2e98474f802171dbf7a61244` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:00.690836 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ccb734-dcf7-428d-9e67-69bbf817ed6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6cfe76f2e98474f802171dbf7a61244)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:00.693442 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ccb734-dcf7-428d-9e67-69bbf817ed6e admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:00.909573 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-d64bfc68-c7ac-479d-99e2-c6f1f554ff58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8db76efa553f4e50ad606dc8713f41b3'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:01.059194 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-d64bfc68-c7ac-479d-99e2-c6f1f554ff58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8db76efa553f4e50ad606dc8713f41b3'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:01.061562 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-d64bfc68-c7ac-479d-99e2-c6f1f554ff58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8db76efa553f4e50ad606dc8713f41b3'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:01.062614 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-d64bfc68-c7ac-479d-99e2-c6f1f554ff58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8db76efa553f4e50ad606dc8713f41b3'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:01.073659 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-87ccb734-dcf7-428d-9e67-69bbf817ed6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6cfe76f2e98474f802171dbf7a61244'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:01.077375 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-d64bfc68-c7ac-479d-99e2-c6f1f554ff58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8db76efa553f4e50ad606dc8713f41b3'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:01.152620 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 852/1713] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:46:00 2022] DELETE /identity/v3/users/8db76efa553f4e50ad606dc8713f41b3 => generated 0 bytes in 764 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:01.154657 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387126e6-fed7-4fed-a477-7acf94831b40 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:01.156447 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387126e6-fed7-4fed-a477-7acf94831b40 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:01.167699 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387126e6-fed7-4fed-a477-7acf94831b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:01.173376 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-87ccb734-dcf7-428d-9e67-69bbf817ed6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6cfe76f2e98474f802171dbf7a61244'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:01.188784 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-87ccb734-dcf7-428d-9e67-69bbf817ed6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6cfe76f2e98474f802171dbf7a61244'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:01.228223 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-87ccb734-dcf7-428d-9e67-69bbf817ed6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6cfe76f2e98474f802171dbf7a61244'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:01.234890 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-87ccb734-dcf7-428d-9e67-69bbf817ed6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6cfe76f2e98474f802171dbf7a61244'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:01.256911 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 862/1714] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:46:00 2022] DELETE /identity/v3/users/d6cfe76f2e98474f802171dbf7a61244 => generated 0 bytes in 790 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:01.263987 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4267cc57-d758-4b6a-9478-e13e7555f082 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:01.264630 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4267cc57-d758-4b6a-9478-e13e7555f082 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:01.265637 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4267cc57-d758-4b6a-9478-e13e7555f082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:01.329679 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-4267cc57-d758-4b6a-9478-e13e7555f082 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:01.336178 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-4267cc57-d758-4b6a-9478-e13e7555f082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:01.370573 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-387126e6-fed7-4fed-a477-7acf94831b40 None None] MFA Rules not processed for user `da799b0044f54da6835aa06e43ce1a6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:01.379198 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-387126e6-fed7-4fed-a477-7acf94831b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:01.514135 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 863/1715] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:01.580309 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4267cc57-d758-4b6a-9478-e13e7555f082 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:01.582013 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-4267cc57-d758-4b6a-9478-e13e7555f082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:01.731671 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 853/1716] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jul 18 16:46:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 578 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:01.796600 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4267cc57-d758-4b6a-9478-e13e7555f082 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:01.801170 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94756f78-ec97-42dc-864d-e0af924280d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:01.802049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94756f78-ec97-42dc-864d-e0af924280d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:01.802492 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94756f78-ec97-42dc-864d-e0af924280d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:01.802925 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94756f78-ec97-42dc-864d-e0af924280d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:01.815955 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94756f78-ec97-42dc-864d-e0af924280d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:01.818132 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94756f78-ec97-42dc-864d-e0af924280d7 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:01.839263 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 864/1717] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:46:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:46:01.976977 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387126e6-fed7-4fed-a477-7acf94831b40 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:01.985324 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-387126e6-fed7-4fed-a477-7acf94831b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:02.025944 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387126e6-fed7-4fed-a477-7acf94831b40 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:02.036051 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c874b6d-d01a-4f2c-bcb1-8407f761e953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:02.040235 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c874b6d-d01a-4f2c-bcb1-8407f761e953 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:02.040235 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c874b6d-d01a-4f2c-bcb1-8407f761e953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:02.040235 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c874b6d-d01a-4f2c-bcb1-8407f761e953 admin admin] PATH_INFO: `/v3/projects/b8766e84cf8d43e0b060e98a39fcf594` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:02.040576 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c874b6d-d01a-4f2c-bcb1-8407f761e953 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8766e84cf8d43e0b060e98a39fcf594)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:02.042552 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c874b6d-d01a-4f2c-bcb1-8407f761e953 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:02.139586 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94756f78-ec97-42dc-864d-e0af924280d7 service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:02.141062 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-94756f78-ec97-42dc-864d-e0af924280d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:02.259031 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-0c874b6d-d01a-4f2c-bcb1-8407f761e953 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8766e84cf8d43e0b060e98a39fcf594', 'request_id': 'req-0c874b6d-d01a-4f2c-bcb1-8407f761e953'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:02.271339 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-0c874b6d-d01a-4f2c-bcb1-8407f761e953 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8766e84cf8d43e0b060e98a39fcf594', 'request_id': 'req-0c874b6d-d01a-4f2c-bcb1-8407f761e953'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:02.381106 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94756f78-ec97-42dc-864d-e0af924280d7 service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:02.415012 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-0c874b6d-d01a-4f2c-bcb1-8407f761e953 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8766e84cf8d43e0b060e98a39fcf594', 'request_id': 'req-0c874b6d-d01a-4f2c-bcb1-8407f761e953'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:02.436837 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-0c874b6d-d01a-4f2c-bcb1-8407f761e953 admin admin] The token cache is being invalidate because project b8766e84cf8d43e0b060e98a39fcf594 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:46:02.437174 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-0c874b6d-d01a-4f2c-bcb1-8407f761e953 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:02.438153 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-0c874b6d-d01a-4f2c-bcb1-8407f761e953 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:02.444048 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 854/1718] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:46:01 2022] DELETE /identity/v3/projects/b8766e84cf8d43e0b060e98a39fcf594 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:02.447379 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9905361-2f98-4273-8ead-ae7edccd8f7f None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:02.447876 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9905361-2f98-4273-8ead-ae7edccd8f7f None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:02.448351 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9905361-2f98-4273-8ead-ae7edccd8f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:02.466925 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:02.501809 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-b9905361-2f98-4273-8ead-ae7edccd8f7f None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:02.523888 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-b9905361-2f98-4273-8ead-ae7edccd8f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:02.526268 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511b10a2-5c4b-43f7-956b-472e478bdfb6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43854d7a6df2466eb7976fabe6011767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:02.527184 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b10a2-5c4b-43f7-956b-472e478bdfb6 service swift] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:02.527663 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b10a2-5c4b-43f7-956b-472e478bdfb6 service swift] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:02.528091 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b10a2-5c4b-43f7-956b-472e478bdfb6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:02.531899 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-511b10a2-5c4b-43f7-956b-472e478bdfb6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:02.662050 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511b10a2-5c4b-43f7-956b-472e478bdfb6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:02.664526 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511b10a2-5c4b-43f7-956b-472e478bdfb6 service swift] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:02.683521 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 855/1719] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:02.694751 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a592cbb2-f667-4803-ba52-27bce427ac48 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:02.696841 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a592cbb2-f667-4803-ba52-27bce427ac48 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:02.730767 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a592cbb2-f667-4803-ba52-27bce427ac48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:02.739810 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 865/1720] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jul 18 16:46:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 581 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 16:46:02.790391 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-a592cbb2-f667-4803-ba52-27bce427ac48 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:02.794926 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-a592cbb2-f667-4803-ba52-27bce427ac48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:02.848424 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511b10a2-5c4b-43f7-956b-472e478bdfb6 service swift] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:02.849313 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 856/1721] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:02.872377 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511b10a2-5c4b-43f7-956b-472e478bdfb6 service swift] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:02.892376 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d9a0c1-a06b-42a6-bb7c-a1a19cb5a8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:02.893118 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9a0c1-a06b-42a6-bb7c-a1a19cb5a8bc admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:02.893410 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9a0c1-a06b-42a6-bb7c-a1a19cb5a8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:02.893694 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9a0c1-a06b-42a6-bb7c-a1a19cb5a8bc admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:02.894452 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d9a0c1-a06b-42a6-bb7c-a1a19cb5a8bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:46:02.894767 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d9a0c1-a06b-42a6-bb7c-a1a19cb5a8bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:02.898102 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d9a0c1-a06b-42a6-bb7c-a1a19cb5a8bc admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:02.951182 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 866/1722] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 16:46:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:46:02.956730 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a592cbb2-f667-4803-ba52-27bce427ac48 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:02.972152 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a592cbb2-f667-4803-ba52-27bce427ac48 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:02.987043 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011bd3f7-2b0d-4079-8f85-086fb5e6f495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:02.990284 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011bd3f7-2b0d-4079-8f85-086fb5e6f495 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:03.001213 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011bd3f7-2b0d-4079-8f85-086fb5e6f495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:03.001213 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011bd3f7-2b0d-4079-8f85-086fb5e6f495 admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:03.001213 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011bd3f7-2b0d-4079-8f85-086fb5e6f495 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:03.001213 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011bd3f7-2b0d-4079-8f85-086fb5e6f495 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:03.094202 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:03.162060 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 857/1723] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:46:02 2022] POST /identity/v3/projects => generated 367 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:46:03.164809 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d9a0c1-a06b-42a6-bb7c-a1a19cb5a8bc admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:03.186912 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d9a0c1-a06b-42a6-bb7c-a1a19cb5a8bc admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:03.390950 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dba396f-c8b1-4681-8378-946d6b355154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:03.391705 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba396f-c8b1-4681-8378-946d6b355154 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:03.391993 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba396f-c8b1-4681-8378-946d6b355154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:03.401192 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba396f-c8b1-4681-8378-946d6b355154 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:03.401192 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dba396f-c8b1-4681-8378-946d6b355154 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:03.401192 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dba396f-c8b1-4681-8378-946d6b355154 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:03.493128 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 867/1724] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:46:03 2022] POST /identity/v3/users => generated 284 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:46:03.496932 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011bd3f7-2b0d-4079-8f85-086fb5e6f495 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:03.520658 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011bd3f7-2b0d-4079-8f85-086fb5e6f495 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:03.539615 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590874a2-d400-4536-a4e6-2ef500094249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:03.544275 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590874a2-d400-4536-a4e6-2ef500094249 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:03.545183 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590874a2-d400-4536-a4e6-2ef500094249 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:03.545907 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590874a2-d400-4536-a4e6-2ef500094249 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:03.546882 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590874a2-d400-4536-a4e6-2ef500094249 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:03.562360 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590874a2-d400-4536-a4e6-2ef500094249 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:03.636557 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 858/1725] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:46:03 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:46:03.641116 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dba396f-c8b1-4681-8378-946d6b355154 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:03.664600 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dba396f-c8b1-4681-8378-946d6b355154 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:03.707914 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b96266b-62c8-4406-9539-efc2f7cf99fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:03.715732 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b96266b-62c8-4406-9539-efc2f7cf99fe admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:03.717348 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b96266b-62c8-4406-9539-efc2f7cf99fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:03.720391 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b96266b-62c8-4406-9539-efc2f7cf99fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:03.731172 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b96266b-62c8-4406-9539-efc2f7cf99fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:03.734842 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b96266b-62c8-4406-9539-efc2f7cf99fe admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:03.795194 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24c3d05-e404-41cc-95eb-12af32ca002f None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:03.797253 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24c3d05-e404-41cc-95eb-12af32ca002f None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:03.797671 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24c3d05-e404-41cc-95eb-12af32ca002f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:03.824518 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 868/1726] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:46:03 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:46:03.826107 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b96266b-62c8-4406-9539-efc2f7cf99fe admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:03.861908 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b96266b-62c8-4406-9539-efc2f7cf99fe admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:03.867403 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-b24c3d05-e404-41cc-95eb-12af32ca002f None None] MFA Rules not processed for user `7a6054dbfbbc487794d2a32061b0ac82`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:03.868922 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-b24c3d05-e404-41cc-95eb-12af32ca002f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:03.875106 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37d4900-c4c6-4ba9-9926-bfeedd3d89f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:03.878179 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37d4900-c4c6-4ba9-9926-bfeedd3d89f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:03.880881 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37d4900-c4c6-4ba9-9926-bfeedd3d89f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:03.881274 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37d4900-c4c6-4ba9-9926-bfeedd3d89f5 admin admin] PATH_INFO: `/v3/projects/ca5099b4cfe347afad488f1ffb8ce1c5/users/74a5dca5b5994840b10bbc9fea5f2f61/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:03.929124 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37d4900-c4c6-4ba9-9926-bfeedd3d89f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca5099b4cfe347afad488f1ffb8ce1c5, user_id=74a5dca5b5994840b10bbc9fea5f2f61, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:03.943042 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37d4900-c4c6-4ba9-9926-bfeedd3d89f5 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:04.095599 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:04.103043 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 869/1727] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:46:03 2022] PUT /identity/v3/projects/ca5099b4cfe347afad488f1ffb8ce1c5/users/74a5dca5b5994840b10bbc9fea5f2f61/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:04.115836 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c23d81-07bb-4218-a3ee-70aaa7079d66 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:04.115836 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c23d81-07bb-4218-a3ee-70aaa7079d66 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:04.115836 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c23d81-07bb-4218-a3ee-70aaa7079d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:04.136263 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 859/1728] 10.222.0.79 () {58 vars in 1015 bytes} [Mon Jul 18 16:46:03 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:46:04.138257 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c059ea79-41d4-410a-bb22-b508c50e35e8 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:04.138805 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c059ea79-41d4-410a-bb22-b508c50e35e8 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:04.139761 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c059ea79-41d4-410a-bb22-b508c50e35e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:04.238553 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-c059ea79-41d4-410a-bb22-b508c50e35e8 None None] MFA Rules not processed for user `74a5dca5b5994840b10bbc9fea5f2f61`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:04.244666 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-c059ea79-41d4-410a-bb22-b508c50e35e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:04.246194 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-97c23d81-07bb-4218-a3ee-70aaa7079d66 None None] MFA Rules not processed for user `642ba7eed36e4956834c06b771259a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:04.255654 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-97c23d81-07bb-4218-a3ee-70aaa7079d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:04.457544 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 860/1729] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:04 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:04.459096 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c059ea79-41d4-410a-bb22-b508c50e35e8 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:04.463631 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-c059ea79-41d4-410a-bb22-b508c50e35e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:04.503405 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 870/1730] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 400 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:04.515477 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c23d81-07bb-4218-a3ee-70aaa7079d66 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:04.623599 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c23d81-07bb-4218-a3ee-70aaa7079d66 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:04.631639 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9524e92f-46a9-46cd-98e8-8fe27d902769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:04.632440 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9524e92f-46a9-46cd-98e8-8fe27d902769 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:04.632772 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9524e92f-46a9-46cd-98e8-8fe27d902769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:04.633055 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9524e92f-46a9-46cd-98e8-8fe27d902769 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:04.633764 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9524e92f-46a9-46cd-98e8-8fe27d902769 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:04.637025 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:04.637427 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9524e92f-46a9-46cd-98e8-8fe27d902769 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:04.705372 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c059ea79-41d4-410a-bb22-b508c50e35e8 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:04.720782 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dac859-2296-4dc6-8b13-d327d1205c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:04.721689 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dac859-2296-4dc6-8b13-d327d1205c49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:04.721980 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dac859-2296-4dc6-8b13-d327d1205c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:04.722260 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dac859-2296-4dc6-8b13-d327d1205c49 admin admin] PATH_INFO: `/v3/projects/1893db2223fc4395a202c5124c94fdf8` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:04.723540 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dac859-2296-4dc6-8b13-d327d1205c49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1893db2223fc4395a202c5124c94fdf8)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:04.725574 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dac859-2296-4dc6-8b13-d327d1205c49 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:05.178129 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 871/1731] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:46:04 2022] POST /identity/v3/projects => generated 367 bytes in 664 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:46:05.180253 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9524e92f-46a9-46cd-98e8-8fe27d902769 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:05.207853 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-9524e92f-46a9-46cd-98e8-8fe27d902769 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:05.225183 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-01dac859-2296-4dc6-8b13-d327d1205c49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1893db2223fc4395a202c5124c94fdf8', 'request_id': 'req-01dac859-2296-4dc6-8b13-d327d1205c49'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:05.230771 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-01dac859-2296-4dc6-8b13-d327d1205c49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1893db2223fc4395a202c5124c94fdf8', 'request_id': 'req-01dac859-2296-4dc6-8b13-d327d1205c49'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:05.323736 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-01dac859-2296-4dc6-8b13-d327d1205c49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1893db2223fc4395a202c5124c94fdf8', 'request_id': 'req-01dac859-2296-4dc6-8b13-d327d1205c49'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:05.345271 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-01dac859-2296-4dc6-8b13-d327d1205c49 admin admin] The token cache is being invalidate because project 1893db2223fc4395a202c5124c94fdf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:46:05.348880 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-01dac859-2296-4dc6-8b13-d327d1205c49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:05.352759 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-01dac859-2296-4dc6-8b13-d327d1205c49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:05.359091 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 861/1732] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:46:04 2022] DELETE /identity/v3/projects/1893db2223fc4395a202c5124c94fdf8 => generated 0 bytes in 901 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:05.368884 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ad8a3-c745-483b-9fd8-01d7b2545a8d None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:05.368884 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ad8a3-c745-483b-9fd8-01d7b2545a8d None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:05.368884 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ad8a3-c745-483b-9fd8-01d7b2545a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:05.391858 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9524e92f-46a9-46cd-98e8-8fe27d902769 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:05.429126 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-ea0ad8a3-c745-483b-9fd8-01d7b2545a8d None None] MFA Rules not processed for user `642ba7eed36e4956834c06b771259a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:05.431103 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-ea0ad8a3-c745-483b-9fd8-01d7b2545a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:05.533159 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7962f4f-8389-48b7-92b0-199361d28a6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:05.533882 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7962f4f-8389-48b7-92b0-199361d28a6f service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:05.537626 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7962f4f-8389-48b7-92b0-199361d28a6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:05.538188 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7962f4f-8389-48b7-92b0-199361d28a6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:05.543198 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-b7962f4f-8389-48b7-92b0-199361d28a6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:05.572661 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 862/1733] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:05.574711 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599b85bf-7835-4a5f-8907-477d389ac3ec None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:05.577122 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7962f4f-8389-48b7-92b0-199361d28a6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:05.586668 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599b85bf-7835-4a5f-8907-477d389ac3ec None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:05.586987 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7962f4f-8389-48b7-92b0-199361d28a6f service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:05.587552 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599b85bf-7835-4a5f-8907-477d389ac3ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:05.635697 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-599b85bf-7835-4a5f-8907-477d389ac3ec None None] MFA Rules not processed for user `c243a50903b244519ccd58f02cb798da`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:05.636184 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 872/1734] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:46:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:46:05.637126 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7962f4f-8389-48b7-92b0-199361d28a6f service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:05.637446 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-599b85bf-7835-4a5f-8907-477d389ac3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:05.640550 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-b7962f4f-8389-48b7-92b0-199361d28a6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:05.770938 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 863/1735] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:05.773543 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f3ee52-27de-4507-9848-fd97b3382da0 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:05.775513 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f3ee52-27de-4507-9848-fd97b3382da0 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:05.776183 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f3ee52-27de-4507-9848-fd97b3382da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:05.815728 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-e0f3ee52-27de-4507-9848-fd97b3382da0 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:05.816971 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-e0f3ee52-27de-4507-9848-fd97b3382da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:05.824534 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7962f4f-8389-48b7-92b0-199361d28a6f service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:05.836977 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 864/1736] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:05.842733 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:05.857673 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7942f1-81c0-485c-b931-0dac786bdfcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:05.858446 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7942f1-81c0-485c-b931-0dac786bdfcf admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:05.858733 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7942f1-81c0-485c-b931-0dac786bdfcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:05.861117 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f3ee52-27de-4507-9848-fd97b3382da0 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:05.863282 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7942f1-81c0-485c-b931-0dac786bdfcf admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:05.864162 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7942f1-81c0-485c-b931-0dac786bdfcf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:05.866963 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-e0f3ee52-27de-4507-9848-fd97b3382da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:05.868668 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7942f1-81c0-485c-b931-0dac786bdfcf admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:06.003656 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 873/1737] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:46:05 2022] POST /identity/v3/users => generated 290 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:46:06.008310 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7942f1-81c0-485c-b931-0dac786bdfcf admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:06.013743 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-8e7942f1-81c0-485c-b931-0dac786bdfcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:06.129821 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f3ee52-27de-4507-9848-fd97b3382da0 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:06.144165 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c935681-b960-44e5-8fdf-008567bc8510 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:06.146823 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c935681-b960-44e5-8fdf-008567bc8510 service nova] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:06.147338 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c935681-b960-44e5-8fdf-008567bc8510 service nova] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:06.147589 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c935681-b960-44e5-8fdf-008567bc8510 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:06.152551 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-6c935681-b960-44e5-8fdf-008567bc8510 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:06.216188 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c935681-b960-44e5-8fdf-008567bc8510 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:06.220879 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c935681-b960-44e5-8fdf-008567bc8510 service nova] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:06.277657 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 865/1738] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:46:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:46:06.281083 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6025ab7-d991-46b8-8c0e-11fb0faab802 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:06.281384 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6025ab7-d991-46b8-8c0e-11fb0faab802 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:06.281670 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6025ab7-d991-46b8-8c0e-11fb0faab802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:06.302065 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7942f1-81c0-485c-b931-0dac786bdfcf admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:06.313261 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:06.317328 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa20995-4c7a-4fa4-bf57-0fd407e377a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:06.319504 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa20995-4c7a-4fa4-bf57-0fd407e377a4 service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:06.320163 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa20995-4c7a-4fa4-bf57-0fd407e377a4 service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:06.320737 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa20995-4c7a-4fa4-bf57-0fd407e377a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:06.336942 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa20995-4c7a-4fa4-bf57-0fd407e377a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:06.339008 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa20995-4c7a-4fa4-bf57-0fd407e377a4 service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:06.356166 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 874/1739] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:46:06 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:46:06.358020 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-f6025ab7-d991-46b8-8c0e-11fb0faab802 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:06.358144 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa20995-4c7a-4fa4-bf57-0fd407e377a4 service nova] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:06.361475 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-f6025ab7-d991-46b8-8c0e-11fb0faab802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:06.374409 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa20995-4c7a-4fa4-bf57-0fd407e377a4 service nova] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:06.380027 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b436e28-52d6-47a3-8592-98bd82cdfbe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:06.385099 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b436e28-52d6-47a3-8592-98bd82cdfbe3 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:06.385099 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b436e28-52d6-47a3-8592-98bd82cdfbe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:06.385099 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b436e28-52d6-47a3-8592-98bd82cdfbe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:06.385099 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b436e28-52d6-47a3-8592-98bd82cdfbe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:06.385099 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b436e28-52d6-47a3-8592-98bd82cdfbe3 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:06.400405 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 866/1740] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:06.403566 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6025ab7-d991-46b8-8c0e-11fb0faab802 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:06.404803 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 875/1741] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:46:06 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:46:06.409660 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b436e28-52d6-47a3-8592-98bd82cdfbe3 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:06.437945 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b436e28-52d6-47a3-8592-98bd82cdfbe3 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:06.443183 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6025ab7-d991-46b8-8c0e-11fb0faab802 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:06.456502 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37de625-f706-487a-b385-60212ee61693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:06.456502 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37de625-f706-487a-b385-60212ee61693 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:06.456772 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37de625-f706-487a-b385-60212ee61693 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:06.458301 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37de625-f706-487a-b385-60212ee61693 admin admin] PATH_INFO: `/v3/projects/e3e3f22b1c394ec494ce01d9f9da8217/users/aa25925e0b79444f91de1606d3b08b3c/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:06.463979 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a59d7c-dd4a-4083-afad-e7ca110c1d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:06.465029 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a59d7c-dd4a-4083-afad-e7ca110c1d3f admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:06.465313 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a59d7c-dd4a-4083-afad-e7ca110c1d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:06.465594 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a59d7c-dd4a-4083-afad-e7ca110c1d3f admin admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:06.466553 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a59d7c-dd4a-4083-afad-e7ca110c1d3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:46:06.466667 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a59d7c-dd4a-4083-afad-e7ca110c1d3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:06.483899 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a59d7c-dd4a-4083-afad-e7ca110c1d3f admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:06.500288 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37de625-f706-487a-b385-60212ee61693 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3e3f22b1c394ec494ce01d9f9da8217, user_id=aa25925e0b79444f91de1606d3b08b3c, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:06.503110 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37de625-f706-487a-b385-60212ee61693 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:06.540560 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 867/1742] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 16:46:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:46:06.545960 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a59d7c-dd4a-4083-afad-e7ca110c1d3f admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:06.608159 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a59d7c-dd4a-4083-afad-e7ca110c1d3f admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:06.630970 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 876/1743] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:46:06 2022] PUT /identity/v3/projects/e3e3f22b1c394ec494ce01d9f9da8217/users/aa25925e0b79444f91de1606d3b08b3c/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:06.637575 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37de625-f706-487a-b385-60212ee61693 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:06.800186 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4c17f3-1cff-4cab-b262-7f99b1b9e27a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:06.800965 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c17f3-1cff-4cab-b262-7f99b1b9e27a admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:06.801293 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c17f3-1cff-4cab-b262-7f99b1b9e27a admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:06.801579 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c17f3-1cff-4cab-b262-7f99b1b9e27a admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:06.802368 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4c17f3-1cff-4cab-b262-7f99b1b9e27a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:06.812523 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4c17f3-1cff-4cab-b262-7f99b1b9e27a admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:06.855604 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37de625-f706-487a-b385-60212ee61693 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:06.874272 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399339b5-616a-41b4-bd6e-895136f76314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:06.875395 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399339b5-616a-41b4-bd6e-895136f76314 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:06.876264 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399339b5-616a-41b4-bd6e-895136f76314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:06.876656 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399339b5-616a-41b4-bd6e-895136f76314 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:06.877626 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399339b5-616a-41b4-bd6e-895136f76314 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:06.878188 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:06.879445 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399339b5-616a-41b4-bd6e-895136f76314 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:06.922124 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 877/1744] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:46:06 2022] GET /identity/v3/roles => generated 1420 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:46:06.932842 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399339b5-616a-41b4-bd6e-895136f76314 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:06.978662 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399339b5-616a-41b4-bd6e-895136f76314 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:07.009606 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d6300-6572-41a2-97a2-9f4cce913e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:07.012475 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d6300-6572-41a2-97a2-9f4cce913e14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:07.012475 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d6300-6572-41a2-97a2-9f4cce913e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:07.012475 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d6300-6572-41a2-97a2-9f4cce913e14 admin admin] PATH_INFO: `/v3/projects/e3e3f22b1c394ec494ce01d9f9da8217/users/aa25925e0b79444f91de1606d3b08b3c/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:07.020031 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 868/1745] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:46:06 2022] POST /identity/v3/projects => generated 373 bytes in 477 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:46:07.022994 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7d6300-6572-41a2-97a2-9f4cce913e14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3e3f22b1c394ec494ce01d9f9da8217, user_id=aa25925e0b79444f91de1606d3b08b3c, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:07.025719 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4c17f3-1cff-4cab-b262-7f99b1b9e27a admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:07.048222 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7d6300-6572-41a2-97a2-9f4cce913e14 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:07.118620 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 878/1746] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:46:06 2022] PUT /identity/v3/projects/e3e3f22b1c394ec494ce01d9f9da8217/users/aa25925e0b79444f91de1606d3b08b3c/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:07.129451 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a9f96-5c76-456c-953a-67101507e7d7 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:07.129896 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a9f96-5c76-456c-953a-67101507e7d7 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:07.130199 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a9f96-5c76-456c-953a-67101507e7d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:07.204670 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-4c0a9f96-5c76-456c-953a-67101507e7d7 None None] MFA Rules not processed for user `aa25925e0b79444f91de1606d3b08b3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:07.206157 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-4c0a9f96-5c76-456c-953a-67101507e7d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:07.384620 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4c17f3-1cff-4cab-b262-7f99b1b9e27a admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:07.522437 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 879/1747] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:07 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 388 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:07.531892 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0a9f96-5c76-456c-953a-67101507e7d7 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:07.539575 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-4c0a9f96-5c76-456c-953a-67101507e7d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:07.656422 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f309d5e7-ce93-4f5d-8b92-ae79a5af56ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:07.658423 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f309d5e7-ce93-4f5d-8b92-ae79a5af56ab admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:07.658927 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f309d5e7-ce93-4f5d-8b92-ae79a5af56ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:07.661112 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f309d5e7-ce93-4f5d-8b92-ae79a5af56ab admin admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:07.669757 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f309d5e7-ce93-4f5d-8b92-ae79a5af56ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:07.677439 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f309d5e7-ce93-4f5d-8b92-ae79a5af56ab admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:07.777779 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0a9f96-5c76-456c-953a-67101507e7d7 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:07.789589 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6f8703-a15e-4aa6-94fd-5baa910da02c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:07.790278 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f8703-a15e-4aa6-94fd-5baa910da02c service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:07.790572 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f8703-a15e-4aa6-94fd-5baa910da02c service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:07.790868 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f8703-a15e-4aa6-94fd-5baa910da02c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:07.821596 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6f8703-a15e-4aa6-94fd-5baa910da02c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:07.822289 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 869/1748] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:46:07 2022] POST /identity/v3/users => generated 287 bytes in 798 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:46:07.823967 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6f8703-a15e-4aa6-94fd-5baa910da02c service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:07.827004 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f309d5e7-ce93-4f5d-8b92-ae79a5af56ab admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:07.851011 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f309d5e7-ce93-4f5d-8b92-ae79a5af56ab admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:07.865068 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c850f264-18a5-4e75-9ac6-37b8707659cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:07.866186 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c850f264-18a5-4e75-9ac6-37b8707659cc admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:07.866636 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c850f264-18a5-4e75-9ac6-37b8707659cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:07.867694 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c850f264-18a5-4e75-9ac6-37b8707659cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:07.868756 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c850f264-18a5-4e75-9ac6-37b8707659cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:07.872308 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c850f264-18a5-4e75-9ac6-37b8707659cc admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:07.881111 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 880/1749] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:46:07 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:46:07.912096 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 870/1750] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:46:07 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:46:07.928350 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6f8703-a15e-4aa6-94fd-5baa910da02c service cinder] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:07.947759 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d673a-1c44-43dc-8ad7-4e553f766c30 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:07.948316 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d673a-1c44-43dc-8ad7-4e553f766c30 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:07.948757 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d673a-1c44-43dc-8ad7-4e553f766c30 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:07.950134 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6f8703-a15e-4aa6-94fd-5baa910da02c service cinder] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:07.950726 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 871/1751] 10.222.0.26 () {54 vars in 912 bytes} [Mon Jul 18 16:46:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:46:07.958119 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed30a3d-ca47-4949-a453-8029b440c920 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:07.958671 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed30a3d-ca47-4949-a453-8029b440c920 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:07.959141 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed30a3d-ca47-4949-a453-8029b440c920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:07.968305 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b676e268-6a10-4217-98c8-89fef3ee9488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:07.969011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b676e268-6a10-4217-98c8-89fef3ee9488 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:07.969298 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b676e268-6a10-4217-98c8-89fef3ee9488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:07.969575 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b676e268-6a10-4217-98c8-89fef3ee9488 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:07.976965 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b676e268-6a10-4217-98c8-89fef3ee9488 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:07.982635 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b676e268-6a10-4217-98c8-89fef3ee9488 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:08.015428 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-fed30a3d-ca47-4949-a453-8029b440c920 None None] MFA Rules not processed for user `7a6054dbfbbc487794d2a32061b0ac82`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:08.017017 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-fed30a3d-ca47-4949-a453-8029b440c920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:08.047998 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 881/1752] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:46:07 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:46:08.053721 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b676e268-6a10-4217-98c8-89fef3ee9488 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:08.080258 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b676e268-6a10-4217-98c8-89fef3ee9488 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:08.105917 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af34583-70f6-46f9-8631-1f3a5da7d4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:08.105917 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af34583-70f6-46f9-8631-1f3a5da7d4c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:08.105917 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af34583-70f6-46f9-8631-1f3a5da7d4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:08.105917 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af34583-70f6-46f9-8631-1f3a5da7d4c0 admin admin] PATH_INFO: `/v3/projects/3b6d4eddd46d470baf8708ff620bc04f/users/ef589cb8f371419381b52f6307521255/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:08.140616 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af34583-70f6-46f9-8631-1f3a5da7d4c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b6d4eddd46d470baf8708ff620bc04f, user_id=ef589cb8f371419381b52f6307521255, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:08.143362 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af34583-70f6-46f9-8631-1f3a5da7d4c0 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:08.247386 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 872/1753] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jul 18 16:46:07 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:46:08.248595 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed30a3d-ca47-4949-a453-8029b440c920 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:08.249997 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-fed30a3d-ca47-4949-a453-8029b440c920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:08.319751 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed30a3d-ca47-4949-a453-8029b440c920 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:08.320002 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 882/1754] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:46:08 2022] PUT /identity/v3/projects/3b6d4eddd46d470baf8708ff620bc04f/users/ef589cb8f371419381b52f6307521255/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:08.321892 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af34583-70f6-46f9-8631-1f3a5da7d4c0 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:08.324971 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-7af34583-70f6-46f9-8631-1f3a5da7d4c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:08.537816 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3afe305-117b-40ba-a61d-8f473dd9975f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:08.544314 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3afe305-117b-40ba-a61d-8f473dd9975f service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:08.544314 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3afe305-117b-40ba-a61d-8f473dd9975f service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:08.545233 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3afe305-117b-40ba-a61d-8f473dd9975f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:08.556193 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-a3afe305-117b-40ba-a61d-8f473dd9975f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:08.565550 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af34583-70f6-46f9-8631-1f3a5da7d4c0 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:08.578348 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482e6cbc-6247-424f-9738-dbf201dba897 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:08.581025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e6cbc-6247-424f-9738-dbf201dba897 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:08.581025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e6cbc-6247-424f-9738-dbf201dba897 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:08.581025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e6cbc-6247-424f-9738-dbf201dba897 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:08.606949 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482e6cbc-6247-424f-9738-dbf201dba897 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:08.608490 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482e6cbc-6247-424f-9738-dbf201dba897 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:08.639588 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 883/1755] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:46:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:46:08.642747 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75896c6f-5e35-4294-ab11-641d25e7307a None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:08.643072 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75896c6f-5e35-4294-ab11-641d25e7307a None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:08.643330 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75896c6f-5e35-4294-ab11-641d25e7307a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:08.734885 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-75896c6f-5e35-4294-ab11-641d25e7307a None None] MFA Rules not processed for user `ef589cb8f371419381b52f6307521255`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:08.736182 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-75896c6f-5e35-4294-ab11-641d25e7307a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:08.746297 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3afe305-117b-40ba-a61d-8f473dd9975f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:08.752203 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3afe305-117b-40ba-a61d-8f473dd9975f service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:08.807460 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 873/1756] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:46:08 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 559 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:46:08.813403 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e49e3c-6bb1-4fcc-99c1-3ed2848291d9 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:08.813673 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e49e3c-6bb1-4fcc-99c1-3ed2848291d9 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:08.817528 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e49e3c-6bb1-4fcc-99c1-3ed2848291d9 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:08.826453 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 874/1757] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jul 18 16:46:08 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:46:08.860262 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:08.870111 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d457fa-af8a-4b6d-8b13-181265266ed6 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:08.870397 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d457fa-af8a-4b6d-8b13-181265266ed6 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:08.870869 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d457fa-af8a-4b6d-8b13-181265266ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:08.967406 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-09d457fa-af8a-4b6d-8b13-181265266ed6 None None] MFA Rules not processed for user `5b868bfa44b94ca5b470166d39f20e54`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:08.975653 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-09d457fa-af8a-4b6d-8b13-181265266ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:09.157108 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 884/1758] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:08 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 513 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:09.403812 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 875/1759] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jul 18 16:46:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 535 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jul 18 16:46:09.589918 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75896c6f-5e35-4294-ab11-641d25e7307a None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:09.612979 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75896c6f-5e35-4294-ab11-641d25e7307a None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:09.631366 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e4850e-b3db-41dd-8493-5ce276bce708 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:09.633375 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4850e-b3db-41dd-8493-5ce276bce708 service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:09.646828 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4850e-b3db-41dd-8493-5ce276bce708 service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:09.649087 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4850e-b3db-41dd-8493-5ce276bce708 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:09.683829 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e4850e-b3db-41dd-8493-5ce276bce708 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:09.688074 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e4850e-b3db-41dd-8493-5ce276bce708 service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:09.702269 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 885/1760] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:46:09 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:46:09.906510 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d457fa-af8a-4b6d-8b13-181265266ed6 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:09.940536 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d457fa-af8a-4b6d-8b13-181265266ed6 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:09.950865 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f44f2-75fa-4565-a071-f3239333d211 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:09.951918 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f44f2-75fa-4565-a071-f3239333d211 service nova] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:09.952383 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f44f2-75fa-4565-a071-f3239333d211 service nova] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:09.952809 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f44f2-75fa-4565-a071-f3239333d211 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:09.968383 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483f44f2-75fa-4565-a071-f3239333d211 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:09.973552 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483f44f2-75fa-4565-a071-f3239333d211 service nova] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:09.999424 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 876/1761] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:46:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:46:10.553185 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a3cb7-be25-4d3e-930a-1019190fe5c8 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:10.553514 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a3cb7-be25-4d3e-930a-1019190fe5c8 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:10.553760 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a3cb7-be25-4d3e-930a-1019190fe5c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:10.607617 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-798a3cb7-be25-4d3e-930a-1019190fe5c8 None None] MFA Rules not processed for user `642ba7eed36e4956834c06b771259a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:10.608369 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-798a3cb7-be25-4d3e-930a-1019190fe5c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:10.649133 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:10.784774 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 886/1762] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:13.151833 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:14.007384 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01981654-f330-4dac-837b-56649865f444 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:14.007903 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01981654-f330-4dac-837b-56649865f444 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:14.008041 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01981654-f330-4dac-837b-56649865f444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:14.053550 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-01981654-f330-4dac-837b-56649865f444 None None] MFA Rules not processed for user `c243a50903b244519ccd58f02cb798da`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:14.055714 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-01981654-f330-4dac-837b-56649865f444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:14.230112 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 877/1763] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:15.135679 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:15.605442 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a3cb7-be25-4d3e-930a-1019190fe5c8 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:15.807923 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a3cb7-be25-4d3e-930a-1019190fe5c8 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:15.826373 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a017431-3e43-4eb0-8d86-f581e96c4022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:15.829762 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a017431-3e43-4eb0-8d86-f581e96c4022 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:15.830057 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a017431-3e43-4eb0-8d86-f581e96c4022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:15.830336 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a017431-3e43-4eb0-8d86-f581e96c4022 admin admin] PATH_INFO: `/v3/users/ef589cb8f371419381b52f6307521255` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:15.832218 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a017431-3e43-4eb0-8d86-f581e96c4022 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef589cb8f371419381b52f6307521255)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:15.838295 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a017431-3e43-4eb0-8d86-f581e96c4022 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:16.188586 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-5a017431-3e43-4eb0-8d86-f581e96c4022 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef589cb8f371419381b52f6307521255'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:16.265370 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-5a017431-3e43-4eb0-8d86-f581e96c4022 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef589cb8f371419381b52f6307521255'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:16.275286 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-5a017431-3e43-4eb0-8d86-f581e96c4022 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef589cb8f371419381b52f6307521255'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:16.297736 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-5a017431-3e43-4eb0-8d86-f581e96c4022 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef589cb8f371419381b52f6307521255'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:16.297736 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-5a017431-3e43-4eb0-8d86-f581e96c4022 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef589cb8f371419381b52f6307521255'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:16.303057 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 887/1764] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:46:15 2022] DELETE /identity/v3/users/ef589cb8f371419381b52f6307521255 => generated 0 bytes in 702 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:16.310462 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a22e6-505d-4fe4-b9ca-dfdc671aa81f None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:16.311257 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a22e6-505d-4fe4-b9ca-dfdc671aa81f None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:16.311520 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a22e6-505d-4fe4-b9ca-dfdc671aa81f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:16.336708 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-004a22e6-505d-4fe4-b9ca-dfdc671aa81f None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:16.338369 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-004a22e6-505d-4fe4-b9ca-dfdc671aa81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:16.426081 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 878/1765] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:16.437146 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a017431-3e43-4eb0-8d86-f581e96c4022 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:16.438664 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-5a017431-3e43-4eb0-8d86-f581e96c4022 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:16.547240 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a017431-3e43-4eb0-8d86-f581e96c4022 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:16.550975 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec38164-a419-4c09-af18-8d479ed6229f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:16.551591 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec38164-a419-4c09-af18-8d479ed6229f service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:16.551873 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec38164-a419-4c09-af18-8d479ed6229f service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:16.552147 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec38164-a419-4c09-af18-8d479ed6229f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:16.562003 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec38164-a419-4c09-af18-8d479ed6229f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:16.563531 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec38164-a419-4c09-af18-8d479ed6229f service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:16.577735 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 888/1766] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:46:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:46:17.013368 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:17.311918 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:17.388350 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004a22e6-505d-4fe4-b9ca-dfdc671aa81f None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:17.390113 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-004a22e6-505d-4fe4-b9ca-dfdc671aa81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:17.406213 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004a22e6-505d-4fe4-b9ca-dfdc671aa81f None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:17.412178 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff27c737-f4fe-4d9e-84f1-899e54e01446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:17.413326 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff27c737-f4fe-4d9e-84f1-899e54e01446 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:17.414356 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff27c737-f4fe-4d9e-84f1-899e54e01446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:17.414778 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff27c737-f4fe-4d9e-84f1-899e54e01446 admin admin] PATH_INFO: `/v3/projects/3b6d4eddd46d470baf8708ff620bc04f` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:17.416603 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff27c737-f4fe-4d9e-84f1-899e54e01446 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b6d4eddd46d470baf8708ff620bc04f)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:17.419566 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff27c737-f4fe-4d9e-84f1-899e54e01446 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:17.608811 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-ff27c737-f4fe-4d9e-84f1-899e54e01446 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b6d4eddd46d470baf8708ff620bc04f', 'request_id': 'req-ff27c737-f4fe-4d9e-84f1-899e54e01446'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:17.615040 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-ff27c737-f4fe-4d9e-84f1-899e54e01446 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b6d4eddd46d470baf8708ff620bc04f', 'request_id': 'req-ff27c737-f4fe-4d9e-84f1-899e54e01446'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:17.730129 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec38164-a419-4c09-af18-8d479ed6229f service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:17.732096 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-dec38164-a419-4c09-af18-8d479ed6229f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:17.748709 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec38164-a419-4c09-af18-8d479ed6229f service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:17.777879 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-ff27c737-f4fe-4d9e-84f1-899e54e01446 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b6d4eddd46d470baf8708ff620bc04f', 'request_id': 'req-ff27c737-f4fe-4d9e-84f1-899e54e01446'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:17.791927 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-ff27c737-f4fe-4d9e-84f1-899e54e01446 admin admin] The token cache is being invalidate because project 3b6d4eddd46d470baf8708ff620bc04f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:46:17.792686 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-ff27c737-f4fe-4d9e-84f1-899e54e01446 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:17.793623 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-ff27c737-f4fe-4d9e-84f1-899e54e01446 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:17.795360 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 879/1767] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:46:17 2022] DELETE /identity/v3/projects/3b6d4eddd46d470baf8708ff620bc04f => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:17.800504 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2aeff4-3cb8-4e80-a3dd-fa844f0f10c1 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:17.801074 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2aeff4-3cb8-4e80-a3dd-fa844f0f10c1 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:17.801358 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2aeff4-3cb8-4e80-a3dd-fa844f0f10c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:17.821667 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715947ad-b24a-4b6f-98b3-eab049f3bcca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:17.822942 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715947ad-b24a-4b6f-98b3-eab049f3bcca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:17.823406 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715947ad-b24a-4b6f-98b3-eab049f3bcca admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:17.823720 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715947ad-b24a-4b6f-98b3-eab049f3bcca admin admin] PATH_INFO: `/v3/users/74a5dca5b5994840b10bbc9fea5f2f61` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:17.824989 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715947ad-b24a-4b6f-98b3-eab049f3bcca admin admin] RBAC: Authorizing `identity:delete_user(user_id=74a5dca5b5994840b10bbc9fea5f2f61)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:17.825689 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-dd2aeff4-3cb8-4e80-a3dd-fa844f0f10c1 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:17.826876 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-dd2aeff4-3cb8-4e80-a3dd-fa844f0f10c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:17.826977 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715947ad-b24a-4b6f-98b3-eab049f3bcca admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:17.835165 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 880/1768] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:46:17.849375 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10216d76-0f69-4e48-9648-273da6978e8b None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:17.849375 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10216d76-0f69-4e48-9648-273da6978e8b None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:17.849375 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10216d76-0f69-4e48-9648-273da6978e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:17.868786 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-10216d76-0f69-4e48-9648-273da6978e8b None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:17.870128 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-10216d76-0f69-4e48-9648-273da6978e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:17.878603 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 881/1769] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:17.883695 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10216d76-0f69-4e48-9648-273da6978e8b None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:17.893000 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10216d76-0f69-4e48-9648-273da6978e8b None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:17.894096 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:17.898172 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed4b522-1f01-4b5f-893d-3bd41f0b3917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:17.899399 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed4b522-1f01-4b5f-893d-3bd41f0b3917 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:17.899852 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed4b522-1f01-4b5f-893d-3bd41f0b3917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:17.900289 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed4b522-1f01-4b5f-893d-3bd41f0b3917 admin admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:17.901023 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed4b522-1f01-4b5f-893d-3bd41f0b3917 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:46:17.904008 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed4b522-1f01-4b5f-893d-3bd41f0b3917 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:17.904008 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed4b522-1f01-4b5f-893d-3bd41f0b3917 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:17.915544 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 882/1770] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 16:46:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:46:17.920216 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed4b522-1f01-4b5f-893d-3bd41f0b3917 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:17.932637 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed4b522-1f01-4b5f-893d-3bd41f0b3917 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:17.937705 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ba6ced-1ef0-4991-bcbf-44b5622f1987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:17.938515 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ba6ced-1ef0-4991-bcbf-44b5622f1987 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:17.938802 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ba6ced-1ef0-4991-bcbf-44b5622f1987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:17.939079 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ba6ced-1ef0-4991-bcbf-44b5622f1987 admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:17.939873 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ba6ced-1ef0-4991-bcbf-44b5622f1987 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:17.941888 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ba6ced-1ef0-4991-bcbf-44b5622f1987 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:18.036920 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 883/1771] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:46:17 2022] POST /identity/v3/projects => generated 363 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:46:18.039650 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-715947ad-b24a-4b6f-98b3-eab049f3bcca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74a5dca5b5994840b10bbc9fea5f2f61'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:18.041864 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ba6ced-1ef0-4991-bcbf-44b5622f1987 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:18.111569 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-715947ad-b24a-4b6f-98b3-eab049f3bcca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74a5dca5b5994840b10bbc9fea5f2f61'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:18.111652 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ba6ced-1ef0-4991-bcbf-44b5622f1987 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:18.125965 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-715947ad-b24a-4b6f-98b3-eab049f3bcca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74a5dca5b5994840b10bbc9fea5f2f61'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:18.136351 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f335730-c2d8-4b7f-b954-be8e101f4071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:18.136351 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f335730-c2d8-4b7f-b954-be8e101f4071 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:18.136351 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f335730-c2d8-4b7f-b954-be8e101f4071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:18.136351 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f335730-c2d8-4b7f-b954-be8e101f4071 admin admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:18.136864 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f335730-c2d8-4b7f-b954-be8e101f4071 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:18.139847 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f335730-c2d8-4b7f-b954-be8e101f4071 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:18.150381 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-715947ad-b24a-4b6f-98b3-eab049f3bcca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74a5dca5b5994840b10bbc9fea5f2f61'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:18.151463 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-715947ad-b24a-4b6f-98b3-eab049f3bcca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74a5dca5b5994840b10bbc9fea5f2f61'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:18.197208 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 889/1772] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:46:17 2022] DELETE /identity/v3/users/74a5dca5b5994840b10bbc9fea5f2f61 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:46:18.198874 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca848f-235f-4134-959b-88add3df6e29 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:18.199660 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca848f-235f-4134-959b-88add3df6e29 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:18.200371 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca848f-235f-4134-959b-88add3df6e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:18.221859 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-e0ca848f-235f-4134-959b-88add3df6e29 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:18.223199 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e0ca848f-235f-4134-959b-88add3df6e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:18.269401 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 884/1773] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:46:18 2022] POST /identity/v3/users => generated 282 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:46:18.270867 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f335730-c2d8-4b7f-b954-be8e101f4071 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:18.273924 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-5f335730-c2d8-4b7f-b954-be8e101f4071 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:18.291639 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f335730-c2d8-4b7f-b954-be8e101f4071 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:18.307547 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 890/1774] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:18.317385 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca848f-235f-4134-959b-88add3df6e29 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:18.319315 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e0ca848f-235f-4134-959b-88add3df6e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:18.385658 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61209b5-1106-4b46-a108-9045a77d4d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:18.387031 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61209b5-1106-4b46-a108-9045a77d4d86 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:18.387256 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61209b5-1106-4b46-a108-9045a77d4d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:18.387582 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61209b5-1106-4b46-a108-9045a77d4d86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:18.388336 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61209b5-1106-4b46-a108-9045a77d4d86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:18.390047 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61209b5-1106-4b46-a108-9045a77d4d86 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:18.407934 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca848f-235f-4134-959b-88add3df6e29 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:18.410270 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 885/1775] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:46:18 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:46:18.417334 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7021542-86e1-4ae7-8952-8c25a2a8d525 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:18.418167 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7021542-86e1-4ae7-8952-8c25a2a8d525 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:18.418549 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7021542-86e1-4ae7-8952-8c25a2a8d525 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:18.418935 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7021542-86e1-4ae7-8952-8c25a2a8d525 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:18.419989 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61209b5-1106-4b46-a108-9045a77d4d86 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:18.441707 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7021542-86e1-4ae7-8952-8c25a2a8d525 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:18.443376 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7021542-86e1-4ae7-8952-8c25a2a8d525 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:18.443526 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61209b5-1106-4b46-a108-9045a77d4d86 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:18.449091 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c71653b-86f8-47ef-969c-40d716172dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:18.449774 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c71653b-86f8-47ef-969c-40d716172dcf admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:18.450227 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c71653b-86f8-47ef-969c-40d716172dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:18.450690 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c71653b-86f8-47ef-969c-40d716172dcf admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:18.451613 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c71653b-86f8-47ef-969c-40d716172dcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:18.453662 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c71653b-86f8-47ef-969c-40d716172dcf admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:18.456309 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 891/1776] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:46:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:46:18.458186 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7021542-86e1-4ae7-8952-8c25a2a8d525 service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:18.459718 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f7021542-86e1-4ae7-8952-8c25a2a8d525 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:18.472067 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 886/1777] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:46:18 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:46:18.478355 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c71653b-86f8-47ef-969c-40d716172dcf admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:18.491086 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7021542-86e1-4ae7-8952-8c25a2a8d525 service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:18.497804 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f3afba-1039-4150-94db-f05ec5b44f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:18.498024 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c71653b-86f8-47ef-969c-40d716172dcf admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:18.498703 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f3afba-1039-4150-94db-f05ec5b44f8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:18.498991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f3afba-1039-4150-94db-f05ec5b44f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:18.499267 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f3afba-1039-4150-94db-f05ec5b44f8e admin admin] PATH_INFO: `/v3/users/9951616cfeee4a2ba8c77d147b5687f8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:18.500596 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f3afba-1039-4150-94db-f05ec5b44f8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9951616cfeee4a2ba8c77d147b5687f8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:18.502759 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f3afba-1039-4150-94db-f05ec5b44f8e admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:18.503805 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b7cfd0-c809-430e-9e2f-71389e32527b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:18.504796 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7cfd0-c809-430e-9e2f-71389e32527b admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:18.505242 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7cfd0-c809-430e-9e2f-71389e32527b admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:18.505520 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7cfd0-c809-430e-9e2f-71389e32527b admin admin] PATH_INFO: `/v3/projects/97eff8f126e34cb78c3f8fb66c5f3222/users/fd17b80bcb40420c9ed221d676bb5222/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:18.523189 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b7cfd0-c809-430e-9e2f-71389e32527b admin admin] RBAC: Authorizing `identity:create_grant(project_id=97eff8f126e34cb78c3f8fb66c5f3222, user_id=fd17b80bcb40420c9ed221d676bb5222, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:18.526107 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b7cfd0-c809-430e-9e2f-71389e32527b admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:18.719816 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 887/1778] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:46:18 2022] PUT /identity/v3/projects/97eff8f126e34cb78c3f8fb66c5f3222/users/fd17b80bcb40420c9ed221d676bb5222/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:18.733214 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d446c5-870a-445b-abee-3718e2fe10f9 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:18.733506 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d446c5-870a-445b-abee-3718e2fe10f9 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:18.733978 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d446c5-870a-445b-abee-3718e2fe10f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:18.769605 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-80d446c5-870a-445b-abee-3718e2fe10f9 None None] MFA Rules not processed for user `fd17b80bcb40420c9ed221d676bb5222`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:18.771406 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-80d446c5-870a-445b-abee-3718e2fe10f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:18.873353 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-d8f3afba-1039-4150-94db-f05ec5b44f8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9951616cfeee4a2ba8c77d147b5687f8'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:18.908827 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 888/1779] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:18 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:46:18.913622 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d446c5-870a-445b-abee-3718e2fe10f9 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:18.985204 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d446c5-870a-445b-abee-3718e2fe10f9 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:18.990307 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1336b74-3fc0-4da8-bbfc-d221efe535d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:18.990990 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1336b74-3fc0-4da8-bbfc-d221efe535d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:18.991267 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1336b74-3fc0-4da8-bbfc-d221efe535d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:18.991575 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1336b74-3fc0-4da8-bbfc-d221efe535d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:18.992496 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1336b74-3fc0-4da8-bbfc-d221efe535d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:18.994592 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1336b74-3fc0-4da8-bbfc-d221efe535d4 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:19.077549 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-d8f3afba-1039-4150-94db-f05ec5b44f8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9951616cfeee4a2ba8c77d147b5687f8'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:19.108269 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-d8f3afba-1039-4150-94db-f05ec5b44f8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9951616cfeee4a2ba8c77d147b5687f8'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:19.139082 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-d8f3afba-1039-4150-94db-f05ec5b44f8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9951616cfeee4a2ba8c77d147b5687f8'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:19.140236 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-d8f3afba-1039-4150-94db-f05ec5b44f8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9951616cfeee4a2ba8c77d147b5687f8'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:19.153339 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 892/1780] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:46:18 2022] DELETE /identity/v3/users/9951616cfeee4a2ba8c77d147b5687f8 => generated 0 bytes in 696 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:19.154446 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f3afba-1039-4150-94db-f05ec5b44f8e admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:19.157780 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d8f3afba-1039-4150-94db-f05ec5b44f8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:19.191822 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 889/1781] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:46:18 2022] POST /identity/v3/projects => generated 365 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:46:19.195742 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc02e4-b845-4688-ae46-3489ebb54bc6 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:19.195742 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc02e4-b845-4688-ae46-3489ebb54bc6 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:19.195742 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc02e4-b845-4688-ae46-3489ebb54bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:19.227386 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-3fdc02e4-b845-4688-ae46-3489ebb54bc6 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:19.229141 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-3fdc02e4-b845-4688-ae46-3489ebb54bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:19.259613 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f3afba-1039-4150-94db-f05ec5b44f8e admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:19.317857 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 890/1782] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:19.319279 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdc02e4-b845-4688-ae46-3489ebb54bc6 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:19.322089 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-3fdc02e4-b845-4688-ae46-3489ebb54bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:19.337602 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b64248-3786-44d5-8812-3f0d45b40fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:19.338486 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b64248-3786-44d5-8812-3f0d45b40fd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:19.338778 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b64248-3786-44d5-8812-3f0d45b40fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:19.347871 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b64248-3786-44d5-8812-3f0d45b40fd4 admin admin] PATH_INFO: `/v3/projects/ca5099b4cfe347afad488f1ffb8ce1c5` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:19.347871 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b64248-3786-44d5-8812-3f0d45b40fd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca5099b4cfe347afad488f1ffb8ce1c5)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:19.348106 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdc02e4-b845-4688-ae46-3489ebb54bc6 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:19.348195 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b64248-3786-44d5-8812-3f0d45b40fd4 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:19.348273 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f11cfb8-e0a1-41ad-acbf-5857d3462de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:19.348449 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11cfb8-e0a1-41ad-acbf-5857d3462de0 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:19.348823 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11cfb8-e0a1-41ad-acbf-5857d3462de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:19.349171 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11cfb8-e0a1-41ad-acbf-5857d3462de0 admin admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:19.349914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f11cfb8-e0a1-41ad-acbf-5857d3462de0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:19.352541 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f11cfb8-e0a1-41ad-acbf-5857d3462de0 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:19.466107 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 891/1783] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:46:19 2022] POST /identity/v3/users => generated 289 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:46:19.467829 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f11cfb8-e0a1-41ad-acbf-5857d3462de0 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:19.470937 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-3f11cfb8-e0a1-41ad-acbf-5857d3462de0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:19.541754 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f11cfb8-e0a1-41ad-acbf-5857d3462de0 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:19.545887 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42da1f59-f04b-4bda-b324-d13f2f5157a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:19.546566 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da1f59-f04b-4bda-b324-d13f2f5157a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:19.546850 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da1f59-f04b-4bda-b324-d13f2f5157a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:19.547126 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da1f59-f04b-4bda-b324-d13f2f5157a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:19.561584 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42da1f59-f04b-4bda-b324-d13f2f5157a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:19.563061 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42da1f59-f04b-4bda-b324-d13f2f5157a8 service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:19.575929 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 892/1784] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:46:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:46:19.578527 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42da1f59-f04b-4bda-b324-d13f2f5157a8 service neutron] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:19.578947 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-94b64248-3786-44d5-8812-3f0d45b40fd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca5099b4cfe347afad488f1ffb8ce1c5', 'request_id': 'req-94b64248-3786-44d5-8812-3f0d45b40fd4'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:19.585219 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-94b64248-3786-44d5-8812-3f0d45b40fd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca5099b4cfe347afad488f1ffb8ce1c5', 'request_id': 'req-94b64248-3786-44d5-8812-3f0d45b40fd4'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:19.590182 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42da1f59-f04b-4bda-b324-d13f2f5157a8 service neutron] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:19.593847 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-94b64248-3786-44d5-8812-3f0d45b40fd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca5099b4cfe347afad488f1ffb8ce1c5', 'request_id': 'req-94b64248-3786-44d5-8812-3f0d45b40fd4'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:19.678738 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03669947-c361-4b26-96df-39b1dffa9202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:19.679546 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03669947-c361-4b26-96df-39b1dffa9202 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:19.681911 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03669947-c361-4b26-96df-39b1dffa9202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:19.683754 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03669947-c361-4b26-96df-39b1dffa9202 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:19.684255 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-94b64248-3786-44d5-8812-3f0d45b40fd4 admin admin] The token cache is being invalidate because project ca5099b4cfe347afad488f1ffb8ce1c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:46:19.684739 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03669947-c361-4b26-96df-39b1dffa9202 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:19.686213 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-94b64248-3786-44d5-8812-3f0d45b40fd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:19.686980 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03669947-c361-4b26-96df-39b1dffa9202 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:19.687099 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-94b64248-3786-44d5-8812-3f0d45b40fd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:19.689030 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 893/1785] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:46:19 2022] DELETE /identity/v3/projects/ca5099b4cfe347afad488f1ffb8ce1c5 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:19.693058 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b64248-3786-44d5-8812-3f0d45b40fd4 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:19.694637 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-94b64248-3786-44d5-8812-3f0d45b40fd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:19.707737 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 893/1786] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:46:19 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:46:19.715753 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03669947-c361-4b26-96df-39b1dffa9202 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:19.718300 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-03669947-c361-4b26-96df-39b1dffa9202 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:19.723510 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b64248-3786-44d5-8812-3f0d45b40fd4 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:19.739176 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44fce21-71df-440e-a3ae-4dd4a10b7f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:19.739176 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44fce21-71df-440e-a3ae-4dd4a10b7f62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:19.739176 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44fce21-71df-440e-a3ae-4dd4a10b7f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:19.739176 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44fce21-71df-440e-a3ae-4dd4a10b7f62 admin admin] PATH_INFO: `/v3/users/aa25925e0b79444f91de1606d3b08b3c` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:19.744742 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44fce21-71df-440e-a3ae-4dd4a10b7f62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa25925e0b79444f91de1606d3b08b3c)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:19.747184 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03669947-c361-4b26-96df-39b1dffa9202 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:19.766085 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c8e401-43a2-422f-b6a9-12af5f458b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:19.767405 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c8e401-43a2-422f-b6a9-12af5f458b6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:19.775972 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c8e401-43a2-422f-b6a9-12af5f458b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:19.776638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c8e401-43a2-422f-b6a9-12af5f458b6a admin admin] PATH_INFO: `/v3/projects/5d00af09b6334ef2b8f7b06fb27c3063/users/1d56e2c90cfd4cfca5714dfefab9bdd3/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:19.780401 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44fce21-71df-440e-a3ae-4dd4a10b7f62 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:19.796561 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c8e401-43a2-422f-b6a9-12af5f458b6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d00af09b6334ef2b8f7b06fb27c3063, user_id=1d56e2c90cfd4cfca5714dfefab9bdd3, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:19.817925 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c8e401-43a2-422f-b6a9-12af5f458b6a admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:20.028858 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 894/1787] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:46:19 2022] PUT /identity/v3/projects/5d00af09b6334ef2b8f7b06fb27c3063/users/1d56e2c90cfd4cfca5714dfefab9bdd3/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:20.034329 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c8e401-43a2-422f-b6a9-12af5f458b6a admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:20.167926 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c8e401-43a2-422f-b6a9-12af5f458b6a admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:20.178515 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c35d619-ce02-40f3-9052-bede535a6b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:20.180653 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c35d619-ce02-40f3-9052-bede535a6b47 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:20.181281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c35d619-ce02-40f3-9052-bede535a6b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:20.181845 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c35d619-ce02-40f3-9052-bede535a6b47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:20.183347 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c35d619-ce02-40f3-9052-bede535a6b47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:20.186594 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c35d619-ce02-40f3-9052-bede535a6b47 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:20.192046 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-e44fce21-71df-440e-a3ae-4dd4a10b7f62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa25925e0b79444f91de1606d3b08b3c'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:20.246271 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 895/1788] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:46:20 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:46:20.250336 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c35d619-ce02-40f3-9052-bede535a6b47 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:20.277941 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c35d619-ce02-40f3-9052-bede535a6b47 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:20.293112 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cbee52-56a3-4ca6-960b-a2e993a2c0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:20.293112 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cbee52-56a3-4ca6-960b-a2e993a2c0a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:20.293112 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cbee52-56a3-4ca6-960b-a2e993a2c0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:20.296213 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cbee52-56a3-4ca6-960b-a2e993a2c0a0 admin admin] PATH_INFO: `/v3/projects/5d00af09b6334ef2b8f7b06fb27c3063/users/1d56e2c90cfd4cfca5714dfefab9bdd3/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:20.305682 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cbee52-56a3-4ca6-960b-a2e993a2c0a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d00af09b6334ef2b8f7b06fb27c3063, user_id=1d56e2c90cfd4cfca5714dfefab9bdd3, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:20.329683 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cbee52-56a3-4ca6-960b-a2e993a2c0a0 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:20.369003 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-e44fce21-71df-440e-a3ae-4dd4a10b7f62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa25925e0b79444f91de1606d3b08b3c'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:20.384842 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-e44fce21-71df-440e-a3ae-4dd4a10b7f62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa25925e0b79444f91de1606d3b08b3c'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:20.384929 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 896/1789] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:46:20 2022] PUT /identity/v3/projects/5d00af09b6334ef2b8f7b06fb27c3063/users/1d56e2c90cfd4cfca5714dfefab9bdd3/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:20.397980 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adaac51-d7c1-4d7c-8443-4e3201289909 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:20.398271 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adaac51-d7c1-4d7c-8443-4e3201289909 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:20.399264 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adaac51-d7c1-4d7c-8443-4e3201289909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:20.409017 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-e44fce21-71df-440e-a3ae-4dd4a10b7f62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa25925e0b79444f91de1606d3b08b3c'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:20.413117 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-e44fce21-71df-440e-a3ae-4dd4a10b7f62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa25925e0b79444f91de1606d3b08b3c'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:20.424182 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 894/1790] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:46:19 2022] DELETE /identity/v3/users/aa25925e0b79444f91de1606d3b08b3c => generated 0 bytes in 732 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:46:20.429894 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44fce21-71df-440e-a3ae-4dd4a10b7f62 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:20.432641 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e44fce21-71df-440e-a3ae-4dd4a10b7f62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:20.489944 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-3adaac51-d7c1-4d7c-8443-4e3201289909 None None] MFA Rules not processed for user `1d56e2c90cfd4cfca5714dfefab9bdd3`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:20.493867 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-3adaac51-d7c1-4d7c-8443-4e3201289909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:20.511630 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:20.583531 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44fce21-71df-440e-a3ae-4dd4a10b7f62 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:20.600173 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b9433f-352a-4942-80db-d20d0f34555f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:20.601127 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9433f-352a-4942-80db-d20d0f34555f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:20.601355 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9433f-352a-4942-80db-d20d0f34555f admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:20.601730 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9433f-352a-4942-80db-d20d0f34555f admin admin] PATH_INFO: `/v3/projects/5d368dc46b5147edbe03a809ed3be56b` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:20.605210 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b9433f-352a-4942-80db-d20d0f34555f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d368dc46b5147edbe03a809ed3be56b)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:20.607151 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b9433f-352a-4942-80db-d20d0f34555f admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:20.677035 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 897/1791] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:20 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:20.701409 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adaac51-d7c1-4d7c-8443-4e3201289909 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:20.703649 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-3adaac51-d7c1-4d7c-8443-4e3201289909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:20.859879 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b3b9433f-352a-4942-80db-d20d0f34555f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d368dc46b5147edbe03a809ed3be56b', 'request_id': 'req-b3b9433f-352a-4942-80db-d20d0f34555f'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:20.885257 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b3b9433f-352a-4942-80db-d20d0f34555f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d368dc46b5147edbe03a809ed3be56b', 'request_id': 'req-b3b9433f-352a-4942-80db-d20d0f34555f'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:20.891150 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b3b9433f-352a-4942-80db-d20d0f34555f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d368dc46b5147edbe03a809ed3be56b', 'request_id': 'req-b3b9433f-352a-4942-80db-d20d0f34555f'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:20.916669 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adaac51-d7c1-4d7c-8443-4e3201289909 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:21.010241 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b3b9433f-352a-4942-80db-d20d0f34555f admin admin] The token cache is being invalidate because project 5d368dc46b5147edbe03a809ed3be56b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:46:21.011036 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b3b9433f-352a-4942-80db-d20d0f34555f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:21.013783 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b3b9433f-352a-4942-80db-d20d0f34555f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:21.019326 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 895/1792] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:46:20 2022] DELETE /identity/v3/projects/5d368dc46b5147edbe03a809ed3be56b => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:46:21.024117 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b9433f-352a-4942-80db-d20d0f34555f admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:21.026528 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-b3b9433f-352a-4942-80db-d20d0f34555f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:21.045332 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d19d3c-3385-458f-bfd5-d54d9b01b09e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:21.048228 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d19d3c-3385-458f-bfd5-d54d9b01b09e service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:21.048650 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d19d3c-3385-458f-bfd5-d54d9b01b09e service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:21.050652 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d19d3c-3385-458f-bfd5-d54d9b01b09e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:21.055928 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-15d19d3c-3385-458f-bfd5-d54d9b01b09e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:21.132396 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:21.166723 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d19d3c-3385-458f-bfd5-d54d9b01b09e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:21.169761 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d19d3c-3385-458f-bfd5-d54d9b01b09e service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:21.199717 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 898/1793] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:46:20 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:46:21.222312 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b9433f-352a-4942-80db-d20d0f34555f admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:21.228334 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0478c8fa-3c82-4c4d-b5d4-6b892c5df3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:21.229712 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0478c8fa-3c82-4c4d-b5d4-6b892c5df3b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:21.230431 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0478c8fa-3c82-4c4d-b5d4-6b892c5df3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:21.230935 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0478c8fa-3c82-4c4d-b5d4-6b892c5df3b0 admin admin] PATH_INFO: `/v3/users/d04f2a5c0a4d48af991e4c1c2df556ed` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:21.233019 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0478c8fa-3c82-4c4d-b5d4-6b892c5df3b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d04f2a5c0a4d48af991e4c1c2df556ed)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:21.235592 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0478c8fa-3c82-4c4d-b5d4-6b892c5df3b0 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:21.468671 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d19d3c-3385-458f-bfd5-d54d9b01b09e service cinder] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:21.470639 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-15d19d3c-3385-458f-bfd5-d54d9b01b09e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:21.491611 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d19d3c-3385-458f-bfd5-d54d9b01b09e service cinder] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:21.494308 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-0478c8fa-3c82-4c4d-b5d4-6b892c5df3b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd04f2a5c0a4d48af991e4c1c2df556ed'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:21.496306 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702705fe-bfd2-4108-b5a7-5a9eb9534afc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:21.497001 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702705fe-bfd2-4108-b5a7-5a9eb9534afc service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:21.497371 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702705fe-bfd2-4108-b5a7-5a9eb9534afc service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:21.500677 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702705fe-bfd2-4108-b5a7-5a9eb9534afc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:21.504830 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-702705fe-bfd2-4108-b5a7-5a9eb9534afc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:21.593181 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-0478c8fa-3c82-4c4d-b5d4-6b892c5df3b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd04f2a5c0a4d48af991e4c1c2df556ed'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:21.605057 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-0478c8fa-3c82-4c4d-b5d4-6b892c5df3b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd04f2a5c0a4d48af991e4c1c2df556ed'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:21.611611 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702705fe-bfd2-4108-b5a7-5a9eb9534afc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:21.613529 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702705fe-bfd2-4108-b5a7-5a9eb9534afc service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:21.626550 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-0478c8fa-3c82-4c4d-b5d4-6b892c5df3b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd04f2a5c0a4d48af991e4c1c2df556ed'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:21.627440 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-0478c8fa-3c82-4c4d-b5d4-6b892c5df3b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd04f2a5c0a4d48af991e4c1c2df556ed'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:21.629773 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 899/1794] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:46:21 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:46:21.642461 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 896/1795] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:46:21 2022] DELETE /identity/v3/users/d04f2a5c0a4d48af991e4c1c2df556ed => generated 0 bytes in 618 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:22.043557 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702705fe-bfd2-4108-b5a7-5a9eb9534afc service cinder] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:22.045114 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-702705fe-bfd2-4108-b5a7-5a9eb9534afc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:22.072399 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702705fe-bfd2-4108-b5a7-5a9eb9534afc service cinder] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:22.208293 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190c29a-51ae-4c1c-86e5-4d4b338ffe6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:22.209826 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190c29a-51ae-4c1c-86e5-4d4b338ffe6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:22.210121 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190c29a-51ae-4c1c-86e5-4d4b338ffe6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:22.210405 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190c29a-51ae-4c1c-86e5-4d4b338ffe6a admin admin] PATH_INFO: `/v3/projects/e3e3f22b1c394ec494ce01d9f9da8217` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:22.222372 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f190c29a-51ae-4c1c-86e5-4d4b338ffe6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3e3f22b1c394ec494ce01d9f9da8217)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:22.224909 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f190c29a-51ae-4c1c-86e5-4d4b338ffe6a admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:22.340671 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:22.529555 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-f190c29a-51ae-4c1c-86e5-4d4b338ffe6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3e3f22b1c394ec494ce01d9f9da8217', 'request_id': 'req-f190c29a-51ae-4c1c-86e5-4d4b338ffe6a'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:22.544759 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1105f46-d4ca-4b05-ade1-ab9ded4f6190 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:22.555147 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1105f46-d4ca-4b05-ade1-ab9ded4f6190 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:22.555147 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1105f46-d4ca-4b05-ade1-ab9ded4f6190 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:22.555147 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 897/1796] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jul 18 16:46:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:46:22.555663 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-f190c29a-51ae-4c1c-86e5-4d4b338ffe6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3e3f22b1c394ec494ce01d9f9da8217', 'request_id': 'req-f190c29a-51ae-4c1c-86e5-4d4b338ffe6a'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:22.571249 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ba9a7-76a8-45bd-99bd-3926e165528f None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:22.571756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ba9a7-76a8-45bd-99bd-3926e165528f None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:22.572480 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ba9a7-76a8-45bd-99bd-3926e165528f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:22.618055 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-bf5ba9a7-76a8-45bd-99bd-3926e165528f None None] MFA Rules not processed for user `5b868bfa44b94ca5b470166d39f20e54`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:22.618055 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-bf5ba9a7-76a8-45bd-99bd-3926e165528f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:22.671537 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-f190c29a-51ae-4c1c-86e5-4d4b338ffe6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3e3f22b1c394ec494ce01d9f9da8217', 'request_id': 'req-f190c29a-51ae-4c1c-86e5-4d4b338ffe6a'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:22.689541 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-f190c29a-51ae-4c1c-86e5-4d4b338ffe6a admin admin] The token cache is being invalidate because project e3e3f22b1c394ec494ce01d9f9da8217 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:46:22.694382 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-f190c29a-51ae-4c1c-86e5-4d4b338ffe6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:22.702762 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-f190c29a-51ae-4c1c-86e5-4d4b338ffe6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:22.714139 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 900/1797] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:46:22 2022] DELETE /identity/v3/projects/e3e3f22b1c394ec494ce01d9f9da8217 => generated 0 bytes in 671 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:22.721240 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190c29a-51ae-4c1c-86e5-4d4b338ffe6a admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:22.722858 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-f190c29a-51ae-4c1c-86e5-4d4b338ffe6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:22.793864 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:22.882352 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 898/1798] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jul 18 16:46:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:22.885251 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef7a6c-64d4-4ad8-9be7-67b999e8aebd None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:22.885955 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef7a6c-64d4-4ad8-9be7-67b999e8aebd None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:22.886656 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef7a6c-64d4-4ad8-9be7-67b999e8aebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:22.903927 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190c29a-51ae-4c1c-86e5-4d4b338ffe6a admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:22.909848 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c85fd7a-625e-4060-9311-02ce6f347a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:22.911358 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c85fd7a-625e-4060-9311-02ce6f347a35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:22.911846 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c85fd7a-625e-4060-9311-02ce6f347a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:22.912279 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c85fd7a-625e-4060-9311-02ce6f347a35 admin admin] PATH_INFO: `/v3/users/3e44347ae1ad434ba9d1f22945125b6d` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:22.915262 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c85fd7a-625e-4060-9311-02ce6f347a35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e44347ae1ad434ba9d1f22945125b6d)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:22.919107 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c85fd7a-625e-4060-9311-02ce6f347a35 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:22.946761 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-e6ef7a6c-64d4-4ad8-9be7-67b999e8aebd None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:22.951429 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e6ef7a6c-64d4-4ad8-9be7-67b999e8aebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:22.976172 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 899/1799] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:22.981850 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ef7a6c-64d4-4ad8-9be7-67b999e8aebd None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:22.987540 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e6ef7a6c-64d4-4ad8-9be7-67b999e8aebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:23.021873 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ef7a6c-64d4-4ad8-9be7-67b999e8aebd None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:23.040808 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3791f227-d4bd-45de-88d7-5931dee1905e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:23.041983 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791f227-d4bd-45de-88d7-5931dee1905e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:23.043093 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791f227-d4bd-45de-88d7-5931dee1905e admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:23.043726 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791f227-d4bd-45de-88d7-5931dee1905e admin admin] PATH_INFO: `/v3/projects/e4cc2c112a8a44e892079f9169bdc3ae` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:23.045467 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3791f227-d4bd-45de-88d7-5931dee1905e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4cc2c112a8a44e892079f9169bdc3ae)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:23.049694 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3791f227-d4bd-45de-88d7-5931dee1905e admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:23.146258 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-9c85fd7a-625e-4060-9311-02ce6f347a35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e44347ae1ad434ba9d1f22945125b6d'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:23.254364 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-9c85fd7a-625e-4060-9311-02ce6f347a35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e44347ae1ad434ba9d1f22945125b6d'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:23.255061 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-9c85fd7a-625e-4060-9311-02ce6f347a35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e44347ae1ad434ba9d1f22945125b6d'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:23.255796 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-9c85fd7a-625e-4060-9311-02ce6f347a35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e44347ae1ad434ba9d1f22945125b6d'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:23.265965 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3791f227-d4bd-45de-88d7-5931dee1905e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4cc2c112a8a44e892079f9169bdc3ae', 'request_id': 'req-3791f227-d4bd-45de-88d7-5931dee1905e'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:23.267123 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-9c85fd7a-625e-4060-9311-02ce6f347a35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e44347ae1ad434ba9d1f22945125b6d'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:23.280580 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3791f227-d4bd-45de-88d7-5931dee1905e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4cc2c112a8a44e892079f9169bdc3ae', 'request_id': 'req-3791f227-d4bd-45de-88d7-5931dee1905e'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:23.295124 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3791f227-d4bd-45de-88d7-5931dee1905e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4cc2c112a8a44e892079f9169bdc3ae', 'request_id': 'req-3791f227-d4bd-45de-88d7-5931dee1905e'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:23.315064 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 901/1800] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:46:22 2022] DELETE /identity/v3/users/3e44347ae1ad434ba9d1f22945125b6d => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:23.320212 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab9746d-cf49-42a4-a241-5615e1091ac6 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:23.321297 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab9746d-cf49-42a4-a241-5615e1091ac6 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:23.321672 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab9746d-cf49-42a4-a241-5615e1091ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:23.344577 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-4ab9746d-cf49-42a4-a241-5615e1091ac6 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:23.345835 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-4ab9746d-cf49-42a4-a241-5615e1091ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:23.414691 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3791f227-d4bd-45de-88d7-5931dee1905e admin admin] The token cache is being invalidate because project e4cc2c112a8a44e892079f9169bdc3ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:46:23.415323 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3791f227-d4bd-45de-88d7-5931dee1905e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:23.416251 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3791f227-d4bd-45de-88d7-5931dee1905e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:23.418412 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 900/1801] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:46:22 2022] DELETE /identity/v3/projects/e4cc2c112a8a44e892079f9169bdc3ae => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:23.425107 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f46e66-e37c-45c7-a281-0f443fd9c646 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:23.425107 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f46e66-e37c-45c7-a281-0f443fd9c646 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:23.425107 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f46e66-e37c-45c7-a281-0f443fd9c646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:23.469402 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 902/1802] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:23.471023 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2bd342-8555-4c6e-a886-2dbda2fe8110 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:23.471317 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2bd342-8555-4c6e-a886-2dbda2fe8110 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:23.473054 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2bd342-8555-4c6e-a886-2dbda2fe8110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:23.487540 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-33f46e66-e37c-45c7-a281-0f443fd9c646 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:23.489078 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-33f46e66-e37c-45c7-a281-0f443fd9c646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:23.501505 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 901/1803] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:23.501505 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f46e66-e37c-45c7-a281-0f443fd9c646 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:23.513102 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f46e66-e37c-45c7-a281-0f443fd9c646 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:23.517068 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f67d0c-374d-43eb-9769-34d4c57c1d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:23.517760 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f67d0c-374d-43eb-9769-34d4c57c1d19 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:23.518045 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f67d0c-374d-43eb-9769-34d4c57c1d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:23.518320 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f67d0c-374d-43eb-9769-34d4c57c1d19 admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:23.519064 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f67d0c-374d-43eb-9769-34d4c57c1d19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:46:23.519403 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f67d0c-374d-43eb-9769-34d4c57c1d19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:23.521073 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f67d0c-374d-43eb-9769-34d4c57c1d19 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:23.522994 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-0a2bd342-8555-4c6e-a886-2dbda2fe8110 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:23.526969 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-0a2bd342-8555-4c6e-a886-2dbda2fe8110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:23.542499 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 902/1804] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 16:46:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:46:23.543589 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f67d0c-374d-43eb-9769-34d4c57c1d19 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:23.546355 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-62f67d0c-374d-43eb-9769-34d4c57c1d19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:23.552125 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 903/1805] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:23.552125 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2bd342-8555-4c6e-a886-2dbda2fe8110 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:23.562133 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-0a2bd342-8555-4c6e-a886-2dbda2fe8110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:23.637324 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f67d0c-374d-43eb-9769-34d4c57c1d19 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:23.643910 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24be075-de8e-4859-bb36-b562e409b7e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:23.644585 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24be075-de8e-4859-bb36-b562e409b7e1 service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:23.644869 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24be075-de8e-4859-bb36-b562e409b7e1 service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:23.645145 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24be075-de8e-4859-bb36-b562e409b7e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:23.647416 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a24be075-de8e-4859-bb36-b562e409b7e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:23.668241 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24be075-de8e-4859-bb36-b562e409b7e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:23.672557 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24be075-de8e-4859-bb36-b562e409b7e1 service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:23.686607 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2bd342-8555-4c6e-a886-2dbda2fe8110 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:23.688743 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 903/1806] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:46:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:46:23.689898 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24be075-de8e-4859-bb36-b562e409b7e1 service nova] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:23.705947 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24be075-de8e-4859-bb36-b562e409b7e1 service nova] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:23.709912 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa50b8a7-c001-436c-a9ea-ecf6c0121666 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:23.712758 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50b8a7-c001-436c-a9ea-ecf6c0121666 service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:23.712758 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50b8a7-c001-436c-a9ea-ecf6c0121666 service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:23.712758 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50b8a7-c001-436c-a9ea-ecf6c0121666 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:23.713279 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2712990a-7670-42a6-932f-0125c97b9383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:23.713279 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712990a-7670-42a6-932f-0125c97b9383 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:23.713993 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712990a-7670-42a6-932f-0125c97b9383 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:23.714835 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712990a-7670-42a6-932f-0125c97b9383 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:23.716222 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2712990a-7670-42a6-932f-0125c97b9383 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:23.718458 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2712990a-7670-42a6-932f-0125c97b9383 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:23.725708 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa50b8a7-c001-436c-a9ea-ecf6c0121666 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:23.729034 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa50b8a7-c001-436c-a9ea-ecf6c0121666 service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:23.742992 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 904/1807] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:46:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:46:23.745824 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5004992-e4af-4201-930e-9539af72bf44 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:23.746343 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5004992-e4af-4201-930e-9539af72bf44 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:23.747154 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5004992-e4af-4201-930e-9539af72bf44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:23.789605 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-c5004992-e4af-4201-930e-9539af72bf44 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:23.791958 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-c5004992-e4af-4201-930e-9539af72bf44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:23.814232 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 905/1808] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:23.815220 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5004992-e4af-4201-930e-9539af72bf44 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:23.828510 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5004992-e4af-4201-930e-9539af72bf44 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:23.833417 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cead9e-d356-4edb-9bdc-900616f4ac0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:23.834155 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cead9e-d356-4edb-9bdc-900616f4ac0b admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:23.834441 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cead9e-d356-4edb-9bdc-900616f4ac0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:23.834715 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cead9e-d356-4edb-9bdc-900616f4ac0b admin admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:23.835562 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cead9e-d356-4edb-9bdc-900616f4ac0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:46:23.835896 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cead9e-d356-4edb-9bdc-900616f4ac0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:23.837797 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cead9e-d356-4edb-9bdc-900616f4ac0b admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:23.848815 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 904/1809] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:46:23 2022] POST /identity/v3/projects => generated 387 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:46:23.849879 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2712990a-7670-42a6-932f-0125c97b9383 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:23.853348 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 906/1810] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 16:46:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:46:23.870207 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cead9e-d356-4edb-9bdc-900616f4ac0b admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:23.887481 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cead9e-d356-4edb-9bdc-900616f4ac0b admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:24.006031 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469dc21d-2954-41cd-993a-5ea6f839e9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:24.006724 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469dc21d-2954-41cd-993a-5ea6f839e9dd admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:24.007032 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469dc21d-2954-41cd-993a-5ea6f839e9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:24.007307 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469dc21d-2954-41cd-993a-5ea6f839e9dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:24.008202 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469dc21d-2954-41cd-993a-5ea6f839e9dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:24.010434 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469dc21d-2954-41cd-993a-5ea6f839e9dd admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:24.013802 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2712990a-7670-42a6-932f-0125c97b9383 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:24.027815 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf273fa-bc38-4ac3-951e-4fe9d926136e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:24.028517 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf273fa-bc38-4ac3-951e-4fe9d926136e admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:24.028806 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf273fa-bc38-4ac3-951e-4fe9d926136e admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:24.029151 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf273fa-bc38-4ac3-951e-4fe9d926136e admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:24.031952 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf273fa-bc38-4ac3-951e-4fe9d926136e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:24.034501 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf273fa-bc38-4ac3-951e-4fe9d926136e admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:24.077870 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:24.153070 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 907/1811] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:46:23 2022] POST /identity/v3/projects => generated 401 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:46:24.154285 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469dc21d-2954-41cd-993a-5ea6f839e9dd admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:24.181703 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 905/1812] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:46:23 2022] POST /identity/v3/users => generated 294 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:46:24.185905 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf273fa-bc38-4ac3-951e-4fe9d926136e admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:24.203460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf273fa-bc38-4ac3-951e-4fe9d926136e admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:24.264638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469dc21d-2954-41cd-993a-5ea6f839e9dd admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:24.293950 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31500da7-83c7-4540-992e-5d10783241b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:24.294709 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31500da7-83c7-4540-992e-5d10783241b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:24.307039 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31500da7-83c7-4540-992e-5d10783241b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:24.307039 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31500da7-83c7-4540-992e-5d10783241b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:24.307039 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31500da7-83c7-4540-992e-5d10783241b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:24.307039 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31500da7-83c7-4540-992e-5d10783241b1 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:24.342291 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6382385e-0911-49ff-9a37-af8c693408f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:24.343076 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6382385e-0911-49ff-9a37-af8c693408f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:24.343428 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6382385e-0911-49ff-9a37-af8c693408f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:24.344037 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6382385e-0911-49ff-9a37-af8c693408f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:24.344903 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6382385e-0911-49ff-9a37-af8c693408f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:24.348009 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6382385e-0911-49ff-9a37-af8c693408f7 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:24.362998 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 906/1813] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:46:24 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:46:24.364279 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6382385e-0911-49ff-9a37-af8c693408f7 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:24.377022 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6382385e-0911-49ff-9a37-af8c693408f7 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:24.391974 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc025b-f46a-437b-b2df-e5c616c5d503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:24.392685 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc025b-f46a-437b-b2df-e5c616c5d503 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:24.393283 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc025b-f46a-437b-b2df-e5c616c5d503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:24.393283 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc025b-f46a-437b-b2df-e5c616c5d503 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:24.394588 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fc025b-f46a-437b-b2df-e5c616c5d503 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:24.395914 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fc025b-f46a-437b-b2df-e5c616c5d503 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:24.437654 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 907/1814] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:46:24 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:46:24.441978 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc025b-f46a-437b-b2df-e5c616c5d503 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:24.458036 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc025b-f46a-437b-b2df-e5c616c5d503 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:24.467031 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056795a3-f912-42ba-9ec1-3653923e7569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:24.469439 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056795a3-f912-42ba-9ec1-3653923e7569 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:24.469964 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056795a3-f912-42ba-9ec1-3653923e7569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:24.470502 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056795a3-f912-42ba-9ec1-3653923e7569 admin admin] PATH_INFO: `/v3/projects/103740daa6b84bd08704436ea2688f20/users/9594992f4a9746a3a489bd26eaa394d4/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:24.499245 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 908/1815] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:46:24 2022] POST /identity/v3/users => generated 301 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:46:24.508609 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31500da7-83c7-4540-992e-5d10783241b1 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:24.522197 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056795a3-f912-42ba-9ec1-3653923e7569 admin admin] RBAC: Authorizing `identity:create_grant(project_id=103740daa6b84bd08704436ea2688f20, user_id=9594992f4a9746a3a489bd26eaa394d4, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:24.528720 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056795a3-f912-42ba-9ec1-3653923e7569 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:24.575776 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31500da7-83c7-4540-992e-5d10783241b1 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:24.598920 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdb0a4b-fd14-4527-ab98-0a32de5f8952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:24.601453 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb0a4b-fd14-4527-ab98-0a32de5f8952 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:24.610972 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb0a4b-fd14-4527-ab98-0a32de5f8952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:24.611222 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb0a4b-fd14-4527-ab98-0a32de5f8952 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:24.619180 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdb0a4b-fd14-4527-ab98-0a32de5f8952 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:24.620901 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdb0a4b-fd14-4527-ab98-0a32de5f8952 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:24.636508 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 908/1816] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:46:24 2022] PUT /identity/v3/projects/103740daa6b84bd08704436ea2688f20/users/9594992f4a9746a3a489bd26eaa394d4/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:24.644612 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 909/1817] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:46:24 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:46:24.645958 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056795a3-f912-42ba-9ec1-3653923e7569 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:24.649362 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ddaca-c098-4e4f-b694-1f97af0aeee1 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:24.649661 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ddaca-c098-4e4f-b694-1f97af0aeee1 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:24.649946 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ddaca-c098-4e4f-b694-1f97af0aeee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:24.710701 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-797ddaca-c098-4e4f-b694-1f97af0aeee1 None None] MFA Rules not processed for user `9594992f4a9746a3a489bd26eaa394d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:24.713066 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-797ddaca-c098-4e4f-b694-1f97af0aeee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:24.777977 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056795a3-f912-42ba-9ec1-3653923e7569 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:24.791863 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f1050-b511-46e7-91e0-e5ec4f7712d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:24.796826 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f1050-b511-46e7-91e0-e5ec4f7712d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:24.796826 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f1050-b511-46e7-91e0-e5ec4f7712d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:24.796826 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f1050-b511-46e7-91e0-e5ec4f7712d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:24.796826 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71f1050-b511-46e7-91e0-e5ec4f7712d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:24.796826 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71f1050-b511-46e7-91e0-e5ec4f7712d8 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:24.866162 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 909/1818] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:46:24 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:46:24.866162 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f1050-b511-46e7-91e0-e5ec4f7712d8 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:24.867170 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 910/1819] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:24.869424 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ddaca-c098-4e4f-b694-1f97af0aeee1 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:24.878408 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f1050-b511-46e7-91e0-e5ec4f7712d8 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:24.882896 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ddaca-c098-4e4f-b694-1f97af0aeee1 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:24.887027 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d9bb36-1a55-46ca-800c-afab0ab36317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:24.887756 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c7b7a-8a7f-4503-9610-127570f492af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:24.888504 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c7b7a-8a7f-4503-9610-127570f492af admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:24.888788 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c7b7a-8a7f-4503-9610-127570f492af admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:24.889181 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c7b7a-8a7f-4503-9610-127570f492af admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:24.889439 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d9bb36-1a55-46ca-800c-afab0ab36317 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:24.889887 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742c7b7a-8a7f-4503-9610-127570f492af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:24.890099 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d9bb36-1a55-46ca-800c-afab0ab36317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:24.891412 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d9bb36-1a55-46ca-800c-afab0ab36317 admin admin] PATH_INFO: `/v3/projects/dab178dc67ed49cb90528aa8910acd52/users/e93df74633a64940bb16b06aae63bfcc/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:24.891822 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742c7b7a-8a7f-4503-9610-127570f492af admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:24.916337 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d9bb36-1a55-46ca-800c-afab0ab36317 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dab178dc67ed49cb90528aa8910acd52, user_id=e93df74633a64940bb16b06aae63bfcc, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:24.922939 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d9bb36-1a55-46ca-800c-afab0ab36317 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:24.952374 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:25.036726 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 911/1820] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:46:24 2022] POST /identity/v3/projects => generated 387 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:46:25.038224 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14391d-c844-402b-a8d5-5e7e256e49b6 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:25.041708 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14391d-c844-402b-a8d5-5e7e256e49b6 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:25.041708 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14391d-c844-402b-a8d5-5e7e256e49b6 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:25.041708 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 912/1821] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jul 18 16:46:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:46:25.049336 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f14391d-c844-402b-a8d5-5e7e256e49b6 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:25.100449 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 910/1822] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:46:24 2022] PUT /identity/v3/projects/dab178dc67ed49cb90528aa8910acd52/users/e93df74633a64940bb16b06aae63bfcc/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:25.102810 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b484d-5f75-4da3-9cb9-d0d30ae80712 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:25.102952 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b484d-5f75-4da3-9cb9-d0d30ae80712 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:25.104121 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b484d-5f75-4da3-9cb9-d0d30ae80712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:25.133416 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-3b8b484d-5f75-4da3-9cb9-d0d30ae80712 None None] MFA Rules not processed for user `5b868bfa44b94ca5b470166d39f20e54`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:25.136127 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-3b8b484d-5f75-4da3-9cb9-d0d30ae80712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:25.288157 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f14391d-c844-402b-a8d5-5e7e256e49b6 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:25.317273 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 911/1823] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jul 18 16:46:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:25.318876 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd80022-c56b-4b17-a938-e7d0ea5a6d34 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:25.319255 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd80022-c56b-4b17-a938-e7d0ea5a6d34 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:25.319948 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd80022-c56b-4b17-a938-e7d0ea5a6d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:25.414931 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-dfd80022-c56b-4b17-a938-e7d0ea5a6d34 None None] MFA Rules not processed for user `e93df74633a64940bb16b06aae63bfcc`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:25.419758 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-dfd80022-c56b-4b17-a938-e7d0ea5a6d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:25.480648 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130870eb-00d9-48a2-aa90-83e545fa3dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:25.481675 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130870eb-00d9-48a2-aa90-83e545fa3dba admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:25.482073 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130870eb-00d9-48a2-aa90-83e545fa3dba admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:25.482430 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130870eb-00d9-48a2-aa90-83e545fa3dba admin admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:25.483456 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130870eb-00d9-48a2-aa90-83e545fa3dba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:25.486715 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130870eb-00d9-48a2-aa90-83e545fa3dba admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:25.544981 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 912/1824] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:25 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:25.544981 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd80022-c56b-4b17-a938-e7d0ea5a6d34 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:25.552028 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-dfd80022-c56b-4b17-a938-e7d0ea5a6d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:25.573561 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd80022-c56b-4b17-a938-e7d0ea5a6d34 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:25.579354 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3741b7d2-2f9e-4f82-be47-a2ba577855a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:25.580641 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3741b7d2-2f9e-4f82-be47-a2ba577855a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:25.581334 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3741b7d2-2f9e-4f82-be47-a2ba577855a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:25.581841 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3741b7d2-2f9e-4f82-be47-a2ba577855a6 admin admin] PATH_INFO: `/v3/projects/32b83bc62e8b44639f7034aa9b269f3e` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:25.584488 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3741b7d2-2f9e-4f82-be47-a2ba577855a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32b83bc62e8b44639f7034aa9b269f3e)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:25.585291 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3741b7d2-2f9e-4f82-be47-a2ba577855a6 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:26.879070 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 913/1825] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:46:25 2022] POST /identity/v3/users => generated 300 bytes in 1830 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:46:26.881028 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130870eb-00d9-48a2-aa90-83e545fa3dba admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:26.884900 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-130870eb-00d9-48a2-aa90-83e545fa3dba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:26.897271 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3741b7d2-2f9e-4f82-be47-a2ba577855a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32b83bc62e8b44639f7034aa9b269f3e', 'request_id': 'req-3741b7d2-2f9e-4f82-be47-a2ba577855a6'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:26.906740 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3741b7d2-2f9e-4f82-be47-a2ba577855a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32b83bc62e8b44639f7034aa9b269f3e', 'request_id': 'req-3741b7d2-2f9e-4f82-be47-a2ba577855a6'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:26.922119 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130870eb-00d9-48a2-aa90-83e545fa3dba admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:26.922892 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3741b7d2-2f9e-4f82-be47-a2ba577855a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32b83bc62e8b44639f7034aa9b269f3e', 'request_id': 'req-3741b7d2-2f9e-4f82-be47-a2ba577855a6'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:27.004501 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc00452e-f4a1-4e3a-b143-90c3dfccda43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:27.005170 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc00452e-f4a1-4e3a-b143-90c3dfccda43 service nova] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:27.005458 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc00452e-f4a1-4e3a-b143-90c3dfccda43 service nova] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:27.005745 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc00452e-f4a1-4e3a-b143-90c3dfccda43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:27.016397 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc00452e-f4a1-4e3a-b143-90c3dfccda43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:27.018015 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc00452e-f4a1-4e3a-b143-90c3dfccda43 service nova] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:27.029962 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 914/1826] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:46:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:46:27.033675 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc00452e-f4a1-4e3a-b143-90c3dfccda43 service nova] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:27.040508 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3741b7d2-2f9e-4f82-be47-a2ba577855a6 admin admin] The token cache is being invalidate because project 32b83bc62e8b44639f7034aa9b269f3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:46:27.040929 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3741b7d2-2f9e-4f82-be47-a2ba577855a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:27.041683 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3741b7d2-2f9e-4f82-be47-a2ba577855a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:27.043369 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 913/1827] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:46:25 2022] DELETE /identity/v3/projects/32b83bc62e8b44639f7034aa9b269f3e => generated 0 bytes in 1499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:27.044780 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3741b7d2-2f9e-4f82-be47-a2ba577855a6 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:27.048171 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-3741b7d2-2f9e-4f82-be47-a2ba577855a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:27.119918 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc00452e-f4a1-4e3a-b143-90c3dfccda43 service nova] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:27.124597 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c183a280-07f1-498d-808f-74cedb9bfd9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:27.125274 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c183a280-07f1-498d-808f-74cedb9bfd9e admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:27.125551 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c183a280-07f1-498d-808f-74cedb9bfd9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:27.125821 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c183a280-07f1-498d-808f-74cedb9bfd9e admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:27.126511 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c183a280-07f1-498d-808f-74cedb9bfd9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:27.127585 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3741b7d2-2f9e-4f82-be47-a2ba577855a6 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:27.128553 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c183a280-07f1-498d-808f-74cedb9bfd9e admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:27.132372 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1028eadc-d561-41a8-a65a-9c79b2a6998d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:27.133931 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028eadc-d561-41a8-a65a-9c79b2a6998d admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:27.134247 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028eadc-d561-41a8-a65a-9c79b2a6998d admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:27.134528 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028eadc-d561-41a8-a65a-9c79b2a6998d admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:27.135784 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1028eadc-d561-41a8-a65a-9c79b2a6998d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:27.136916 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1028eadc-d561-41a8-a65a-9c79b2a6998d admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:27.147355 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 914/1828] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:46:27 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:46:27.148374 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1028eadc-d561-41a8-a65a-9c79b2a6998d admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:27.150068 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-1028eadc-d561-41a8-a65a-9c79b2a6998d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:27.162876 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1028eadc-d561-41a8-a65a-9c79b2a6998d admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:27.167595 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff2716-1103-4893-9a44-06dc786ec152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:27.168492 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff2716-1103-4893-9a44-06dc786ec152 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:27.168779 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff2716-1103-4893-9a44-06dc786ec152 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:27.169056 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff2716-1103-4893-9a44-06dc786ec152 admin admin] PATH_INFO: `/v3/users/1e37d08c5c594a9abf7e8c2b6f2f7432` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:27.170272 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ff2716-1103-4893-9a44-06dc786ec152 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e37d08c5c594a9abf7e8c2b6f2f7432)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:27.172305 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ff2716-1103-4893-9a44-06dc786ec152 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:27.259045 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 915/1829] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:46:27 2022] POST /identity/v3/projects => generated 401 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:46:27.260097 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c183a280-07f1-498d-808f-74cedb9bfd9e admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:27.332643 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c183a280-07f1-498d-808f-74cedb9bfd9e admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:27.338098 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661a9876-e6b5-4b5c-bfc3-5fac1393a4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:27.339013 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661a9876-e6b5-4b5c-bfc3-5fac1393a4e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:27.339295 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661a9876-e6b5-4b5c-bfc3-5fac1393a4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:27.339601 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661a9876-e6b5-4b5c-bfc3-5fac1393a4e9 admin admin] PATH_INFO: `/v3/projects/281ade76a03b4ee48bd58dacce86c2ab/users/2c54e3cc5cb241beb6b4e4d3e4fb1c51/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:27.355372 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661a9876-e6b5-4b5c-bfc3-5fac1393a4e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=281ade76a03b4ee48bd58dacce86c2ab, user_id=2c54e3cc5cb241beb6b4e4d3e4fb1c51, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:27.358360 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661a9876-e6b5-4b5c-bfc3-5fac1393a4e9 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:27.468174 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 916/1830] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:46:27 2022] PUT /identity/v3/projects/281ade76a03b4ee48bd58dacce86c2ab/users/2c54e3cc5cb241beb6b4e4d3e4fb1c51/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:27.469418 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661a9876-e6b5-4b5c-bfc3-5fac1393a4e9 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:27.470968 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-661a9876-e6b5-4b5c-bfc3-5fac1393a4e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:27.476245 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b1ff2716-1103-4893-9a44-06dc786ec152 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e37d08c5c594a9abf7e8c2b6f2f7432'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:27.540478 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661a9876-e6b5-4b5c-bfc3-5fac1393a4e9 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:27.550068 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18975bac-adf9-4655-a25e-e8d323fd23bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:27.550068 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18975bac-adf9-4655-a25e-e8d323fd23bf admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:27.550068 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18975bac-adf9-4655-a25e-e8d323fd23bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:27.550068 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18975bac-adf9-4655-a25e-e8d323fd23bf admin admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:27.550551 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18975bac-adf9-4655-a25e-e8d323fd23bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:27.550551 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18975bac-adf9-4655-a25e-e8d323fd23bf admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:27.587156 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b1ff2716-1103-4893-9a44-06dc786ec152 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e37d08c5c594a9abf7e8c2b6f2f7432'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:27.597489 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b1ff2716-1103-4893-9a44-06dc786ec152 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e37d08c5c594a9abf7e8c2b6f2f7432'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:27.617369 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b1ff2716-1103-4893-9a44-06dc786ec152 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e37d08c5c594a9abf7e8c2b6f2f7432'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:27.619206 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b1ff2716-1103-4893-9a44-06dc786ec152 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e37d08c5c594a9abf7e8c2b6f2f7432'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:27.629364 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 915/1831] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:46:27 2022] DELETE /identity/v3/users/1e37d08c5c594a9abf7e8c2b6f2f7432 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:27.631327 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff2716-1103-4893-9a44-06dc786ec152 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:27.635418 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-b1ff2716-1103-4893-9a44-06dc786ec152 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:27.698794 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 917/1832] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:46:27 2022] POST /identity/v3/users => generated 307 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:46:27.706269 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18975bac-adf9-4655-a25e-e8d323fd23bf admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:27.708176 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-18975bac-adf9-4655-a25e-e8d323fd23bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:27.725183 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18975bac-adf9-4655-a25e-e8d323fd23bf admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:27.746962 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff2716-1103-4893-9a44-06dc786ec152 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:27.754030 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd85b78a-fbbf-4e93-86e1-0137ce74fdc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:27.755814 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd85b78a-fbbf-4e93-86e1-0137ce74fdc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:27.756112 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd85b78a-fbbf-4e93-86e1-0137ce74fdc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:27.756395 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd85b78a-fbbf-4e93-86e1-0137ce74fdc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:27.757778 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd85b78a-fbbf-4e93-86e1-0137ce74fdc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:27.761022 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd85b78a-fbbf-4e93-86e1-0137ce74fdc9 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:27.773385 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 916/1833] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:46:27 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:46:27.778707 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd85b78a-fbbf-4e93-86e1-0137ce74fdc9 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:27.790963 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd85b78a-fbbf-4e93-86e1-0137ce74fdc9 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:27.795774 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f765828c-ca72-4cec-989f-566833e76c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:27.796675 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765828c-ca72-4cec-989f-566833e76c04 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:27.796961 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765828c-ca72-4cec-989f-566833e76c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:27.797242 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765828c-ca72-4cec-989f-566833e76c04 admin admin] PATH_INFO: `/v3/projects/281ade76a03b4ee48bd58dacce86c2ab/users/2c54e3cc5cb241beb6b4e4d3e4fb1c51/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:27.799368 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f765828c-ca72-4cec-989f-566833e76c04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=281ade76a03b4ee48bd58dacce86c2ab, user_id=2c54e3cc5cb241beb6b4e4d3e4fb1c51, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:27.802322 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f765828c-ca72-4cec-989f-566833e76c04 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:27.802442 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd91426-52bc-4343-99dc-861671529d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:27.803040 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd91426-52bc-4343-99dc-861671529d3c admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:27.803323 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd91426-52bc-4343-99dc-861671529d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:27.803718 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd91426-52bc-4343-99dc-861671529d3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:27.804371 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd91426-52bc-4343-99dc-861671529d3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:27.809470 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd91426-52bc-4343-99dc-861671529d3c admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:27.814240 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 917/1834] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:46:27 2022] PUT /identity/v3/projects/281ade76a03b4ee48bd58dacce86c2ab/users/2c54e3cc5cb241beb6b4e4d3e4fb1c51/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:27.821033 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 918/1835] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:46:27 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:46:27.822334 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f765828c-ca72-4cec-989f-566833e76c04 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:27.826086 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d9dae-3639-478a-b9ed-f46661d09b14 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:27.826378 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d9dae-3639-478a-b9ed-f46661d09b14 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:27.826660 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d9dae-3639-478a-b9ed-f46661d09b14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:27.857700 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-857d9dae-3639-478a-b9ed-f46661d09b14 None None] MFA Rules not processed for user `2c54e3cc5cb241beb6b4e4d3e4fb1c51`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:27.858928 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-857d9dae-3639-478a-b9ed-f46661d09b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:27.866423 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:27.915545 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f765828c-ca72-4cec-989f-566833e76c04 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:27.926408 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3f7281-a7f7-4136-919b-6ed4805f5490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:27.927743 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f7281-a7f7-4136-919b-6ed4805f5490 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:27.928032 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f7281-a7f7-4136-919b-6ed4805f5490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:27.928311 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f7281-a7f7-4136-919b-6ed4805f5490 admin admin] PATH_INFO: `/v3/projects/f45ab42b014b4523b145fe60a38b467f/users/f3194adc8f764b11ba06b1ec41eace43/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:27.948264 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3f7281-a7f7-4136-919b-6ed4805f5490 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f45ab42b014b4523b145fe60a38b467f, user_id=f3194adc8f764b11ba06b1ec41eace43, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:27.954779 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 919/1836] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:27 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:46:27.955414 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3f7281-a7f7-4136-919b-6ed4805f5490 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:27.964091 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857d9dae-3639-478a-b9ed-f46661d09b14 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:27.969872 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-857d9dae-3639-478a-b9ed-f46661d09b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:28.028260 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 918/1837] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:46:27 2022] PUT /identity/v3/projects/f45ab42b014b4523b145fe60a38b467f/users/f3194adc8f764b11ba06b1ec41eace43/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:28.034941 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3f7281-a7f7-4136-919b-6ed4805f5490 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:28.093742 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857d9dae-3639-478a-b9ed-f46661d09b14 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:28.152120 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3f7281-a7f7-4136-919b-6ed4805f5490 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:28.167549 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083019bf-580e-4677-89a8-097ba103da5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:28.168379 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083019bf-580e-4677-89a8-097ba103da5c admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:28.168694 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083019bf-580e-4677-89a8-097ba103da5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:28.168955 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083019bf-580e-4677-89a8-097ba103da5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:28.169762 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083019bf-580e-4677-89a8-097ba103da5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:28.171724 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083019bf-580e-4677-89a8-097ba103da5c admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:28.195206 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d65d4df-d1fd-498e-a34b-f86c5fdb2feb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:28.196768 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65d4df-d1fd-498e-a34b-f86c5fdb2feb service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:28.197071 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65d4df-d1fd-498e-a34b-f86c5fdb2feb service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:28.197430 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65d4df-d1fd-498e-a34b-f86c5fdb2feb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:28.197998 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 919/1838] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:46:28 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:46:28.199003 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083019bf-580e-4677-89a8-097ba103da5c admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:28.221968 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083019bf-580e-4677-89a8-097ba103da5c admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:28.231349 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee20de8-c7c0-4e22-862c-5baae08faa80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:28.232383 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee20de8-c7c0-4e22-862c-5baae08faa80 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:28.232669 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee20de8-c7c0-4e22-862c-5baae08faa80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:28.233018 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee20de8-c7c0-4e22-862c-5baae08faa80 admin admin] PATH_INFO: `/v3/projects/f45ab42b014b4523b145fe60a38b467f/users/f3194adc8f764b11ba06b1ec41eace43/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:28.236107 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee20de8-c7c0-4e22-862c-5baae08faa80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f45ab42b014b4523b145fe60a38b467f, user_id=f3194adc8f764b11ba06b1ec41eace43, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:28.239454 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee20de8-c7c0-4e22-862c-5baae08faa80 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:28.272146 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 920/1839] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:46:28 2022] PUT /identity/v3/projects/f45ab42b014b4523b145fe60a38b467f/users/f3194adc8f764b11ba06b1ec41eace43/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:28.279989 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee20de8-c7c0-4e22-862c-5baae08faa80 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:28.281852 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-aee20de8-c7c0-4e22-862c-5baae08faa80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:28.314951 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d65d4df-d1fd-498e-a34b-f86c5fdb2feb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:28.316630 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d65d4df-d1fd-498e-a34b-f86c5fdb2feb service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:28.329433 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 920/1840] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:46:27 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:46:28.330901 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf29935-b3b8-42d8-a1f4-ebaf58d8a7a8 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:28.331332 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf29935-b3b8-42d8-a1f4-ebaf58d8a7a8 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:28.331632 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf29935-b3b8-42d8-a1f4-ebaf58d8a7a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:28.373903 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-7bf29935-b3b8-42d8-a1f4-ebaf58d8a7a8 None None] MFA Rules not processed for user `f3194adc8f764b11ba06b1ec41eace43`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:28.375394 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-7bf29935-b3b8-42d8-a1f4-ebaf58d8a7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:28.381297 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee20de8-c7c0-4e22-862c-5baae08faa80 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:28.390907 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b499126-a441-42e4-b013-c2c00a44dcad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:28.390907 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b499126-a441-42e4-b013-c2c00a44dcad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:28.390907 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b499126-a441-42e4-b013-c2c00a44dcad admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:28.390907 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b499126-a441-42e4-b013-c2c00a44dcad admin admin] PATH_INFO: `/v3/projects/50eba0987a1d4aa0b741d7bcac564c38` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:28.392904 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b499126-a441-42e4-b013-c2c00a44dcad admin admin] RBAC: Authorizing `identity:delete_project(project_id=50eba0987a1d4aa0b741d7bcac564c38)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:28.394944 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b499126-a441-42e4-b013-c2c00a44dcad admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:28.470163 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 921/1841] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:28 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:28.471019 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf29935-b3b8-42d8-a1f4-ebaf58d8a7a8 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:28.487731 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf29935-b3b8-42d8-a1f4-ebaf58d8a7a8 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:28.494020 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574389d5-76f3-49e1-8a92-3d1330d3d13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:28.494932 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574389d5-76f3-49e1-8a92-3d1330d3d13c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:28.495270 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574389d5-76f3-49e1-8a92-3d1330d3d13c admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:28.495584 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574389d5-76f3-49e1-8a92-3d1330d3d13c admin admin] PATH_INFO: `/v3/users/9594992f4a9746a3a489bd26eaa394d4` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:28.496907 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574389d5-76f3-49e1-8a92-3d1330d3d13c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9594992f4a9746a3a489bd26eaa394d4)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:28.498912 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574389d5-76f3-49e1-8a92-3d1330d3d13c admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:28.542748 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-6b499126-a441-42e4-b013-c2c00a44dcad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50eba0987a1d4aa0b741d7bcac564c38', 'request_id': 'req-6b499126-a441-42e4-b013-c2c00a44dcad'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:28.554173 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-6b499126-a441-42e4-b013-c2c00a44dcad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50eba0987a1d4aa0b741d7bcac564c38', 'request_id': 'req-6b499126-a441-42e4-b013-c2c00a44dcad'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:28.565232 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-6b499126-a441-42e4-b013-c2c00a44dcad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50eba0987a1d4aa0b741d7bcac564c38', 'request_id': 'req-6b499126-a441-42e4-b013-c2c00a44dcad'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:28.717499 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-6b499126-a441-42e4-b013-c2c00a44dcad admin admin] The token cache is being invalidate because project 50eba0987a1d4aa0b741d7bcac564c38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:46:28.717744 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-6b499126-a441-42e4-b013-c2c00a44dcad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:28.720391 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-6b499126-a441-42e4-b013-c2c00a44dcad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:28.724349 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 921/1842] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:46:28 2022] DELETE /identity/v3/projects/50eba0987a1d4aa0b741d7bcac564c38 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:28.732639 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833fd09-0d7e-4ab9-98bd-2496d91100c8 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:28.732929 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833fd09-0d7e-4ab9-98bd-2496d91100c8 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:28.733213 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833fd09-0d7e-4ab9-98bd-2496d91100c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:28.755276 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-e833fd09-0d7e-4ab9-98bd-2496d91100c8 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:28.756809 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e833fd09-0d7e-4ab9-98bd-2496d91100c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:28.783677 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-574389d5-76f3-49e1-8a92-3d1330d3d13c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9594992f4a9746a3a489bd26eaa394d4'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:28.830212 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 922/1843] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:28.840106 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a986f5cb-acad-469c-a026-c26484bb8fbb None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:28.840399 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a986f5cb-acad-469c-a026-c26484bb8fbb None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:28.840679 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a986f5cb-acad-469c-a026-c26484bb8fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:28.869642 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-a986f5cb-acad-469c-a026-c26484bb8fbb None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:28.870965 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a986f5cb-acad-469c-a026-c26484bb8fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:28.882783 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 923/1844] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:28.885541 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a986f5cb-acad-469c-a026-c26484bb8fbb None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:28.895483 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a986f5cb-acad-469c-a026-c26484bb8fbb None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:28.901056 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-574389d5-76f3-49e1-8a92-3d1330d3d13c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9594992f4a9746a3a489bd26eaa394d4'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:28.901328 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9e781f-1be3-4e5a-905c-1a07aa9153af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:28.901959 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-574389d5-76f3-49e1-8a92-3d1330d3d13c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9594992f4a9746a3a489bd26eaa394d4'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:28.908219 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e781f-1be3-4e5a-905c-1a07aa9153af admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:28.908219 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e781f-1be3-4e5a-905c-1a07aa9153af admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:28.908702 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-574389d5-76f3-49e1-8a92-3d1330d3d13c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9594992f4a9746a3a489bd26eaa394d4'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:28.908791 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e781f-1be3-4e5a-905c-1a07aa9153af admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:28.908791 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9e781f-1be3-4e5a-905c-1a07aa9153af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:46:28.908791 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9e781f-1be3-4e5a-905c-1a07aa9153af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:28.908791 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9e781f-1be3-4e5a-905c-1a07aa9153af admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:28.914281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-574389d5-76f3-49e1-8a92-3d1330d3d13c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9594992f4a9746a3a489bd26eaa394d4'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:28.918956 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 924/1845] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 16:46:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:46:28.920004 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9e781f-1be3-4e5a-905c-1a07aa9153af admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:28.921819 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-db9e781f-1be3-4e5a-905c-1a07aa9153af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:28.934736 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:28.940169 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 922/1846] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:46:28 2022] DELETE /identity/v3/users/9594992f4a9746a3a489bd26eaa394d4 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:28.944628 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574389d5-76f3-49e1-8a92-3d1330d3d13c admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:28.948074 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-574389d5-76f3-49e1-8a92-3d1330d3d13c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:28.996117 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9e781f-1be3-4e5a-905c-1a07aa9153af admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:29.050796 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574389d5-76f3-49e1-8a92-3d1330d3d13c admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:29.056424 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4619f37-172d-46a0-9d57-7631f1d36f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:29.057234 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4619f37-172d-46a0-9d57-7631f1d36f1e admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:29.057624 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4619f37-172d-46a0-9d57-7631f1d36f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:29.058024 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4619f37-172d-46a0-9d57-7631f1d36f1e admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:29.058888 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4619f37-172d-46a0-9d57-7631f1d36f1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:29.061193 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4619f37-172d-46a0-9d57-7631f1d36f1e admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:29.064381 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138215f2-9a2e-4ab8-8db9-f5c0853cd8f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:29.065054 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138215f2-9a2e-4ab8-8db9-f5c0853cd8f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:29.065343 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138215f2-9a2e-4ab8-8db9-f5c0853cd8f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:29.065622 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138215f2-9a2e-4ab8-8db9-f5c0853cd8f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:29.067876 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-138215f2-9a2e-4ab8-8db9-f5c0853cd8f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:29.154810 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138215f2-9a2e-4ab8-8db9-f5c0853cd8f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:29.162760 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138215f2-9a2e-4ab8-8db9-f5c0853cd8f8 service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:29.167397 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 923/1847] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:46:28 2022] POST /identity/v3/projects => generated 369 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:46:29.170146 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ebdf4-253a-4189-a99c-86f808bc65eb None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:29.170515 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ebdf4-253a-4189-a99c-86f808bc65eb None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:29.170881 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ebdf4-253a-4189-a99c-86f808bc65eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:29.174789 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 925/1848] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:46:28 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:46:29.175657 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138215f2-9a2e-4ab8-8db9-f5c0853cd8f8 service cinder] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:29.188631 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138215f2-9a2e-4ab8-8db9-f5c0853cd8f8 service cinder] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:29.197454 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-ed4ebdf4-253a-4189-a99c-86f808bc65eb None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:29.199081 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-ed4ebdf4-253a-4189-a99c-86f808bc65eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:29.267756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a151b1-2501-4eb6-9c1a-e8f9c25fefd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:29.268456 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a151b1-2501-4eb6-9c1a-e8f9c25fefd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:29.268754 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a151b1-2501-4eb6-9c1a-e8f9c25fefd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:29.269083 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a151b1-2501-4eb6-9c1a-e8f9c25fefd3 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:29.269813 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a151b1-2501-4eb6-9c1a-e8f9c25fefd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:29.271970 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a151b1-2501-4eb6-9c1a-e8f9c25fefd3 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:29.292918 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 924/1849] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:29.303357 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4ebdf4-253a-4189-a99c-86f808bc65eb None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:29.308187 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-ed4ebdf4-253a-4189-a99c-86f808bc65eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:29.378927 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 926/1850] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:46:29 2022] POST /identity/v3/users => generated 285 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:46:29.378927 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a151b1-2501-4eb6-9c1a-e8f9c25fefd3 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:29.388668 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a151b1-2501-4eb6-9c1a-e8f9c25fefd3 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:29.396617 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c63b742-71cf-438d-8473-7303cc1a892d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:29.397928 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63b742-71cf-438d-8473-7303cc1a892d admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:29.398370 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63b742-71cf-438d-8473-7303cc1a892d admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:29.399255 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63b742-71cf-438d-8473-7303cc1a892d admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:29.400325 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c63b742-71cf-438d-8473-7303cc1a892d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:29.402720 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c63b742-71cf-438d-8473-7303cc1a892d admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:29.411271 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4ebdf4-253a-4189-a99c-86f808bc65eb None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:29.416218 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe2b9b-a645-447f-8c11-25e7d6384810 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:29.417092 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe2b9b-a645-447f-8c11-25e7d6384810 service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:29.417327 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 927/1851] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:46:29 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:46:29.417747 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe2b9b-a645-447f-8c11-25e7d6384810 service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:29.418032 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe2b9b-a645-447f-8c11-25e7d6384810 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:29.421520 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c63b742-71cf-438d-8473-7303cc1a892d admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:29.429069 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fe2b9b-a645-447f-8c11-25e7d6384810 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:29.430966 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fe2b9b-a645-447f-8c11-25e7d6384810 service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:29.434055 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c63b742-71cf-438d-8473-7303cc1a892d admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:29.439235 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c9e320-36df-438f-8f21-cbea2e118ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:29.446784 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c9e320-36df-438f-8f21-cbea2e118ffa admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:29.446784 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c9e320-36df-438f-8f21-cbea2e118ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:29.446784 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c9e320-36df-438f-8f21-cbea2e118ffa admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:29.446784 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c9e320-36df-438f-8f21-cbea2e118ffa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:29.447110 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 925/1852] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:46:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:46:29.447196 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c9e320-36df-438f-8f21-cbea2e118ffa admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:29.455545 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 928/1853] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:46:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:46:29.459655 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe2b9b-a645-447f-8c11-25e7d6384810 service neutron] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:29.470521 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe2b9b-a645-447f-8c11-25e7d6384810 service neutron] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:29.475625 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0558b3f-28fa-4314-8bb7-f3e907490def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:29.476538 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0558b3f-28fa-4314-8bb7-f3e907490def admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:29.477403 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0558b3f-28fa-4314-8bb7-f3e907490def admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:29.477686 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0558b3f-28fa-4314-8bb7-f3e907490def admin admin] PATH_INFO: `/v3/projects/56487a7b37614439a9245f4ebad97432/users/5ff5318ef7ae4f31885a53b79de2f3f5/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:29.493443 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0558b3f-28fa-4314-8bb7-f3e907490def admin admin] RBAC: Authorizing `identity:create_grant(project_id=56487a7b37614439a9245f4ebad97432, user_id=5ff5318ef7ae4f31885a53b79de2f3f5, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:29.496249 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0558b3f-28fa-4314-8bb7-f3e907490def admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:29.582424 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 926/1854] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:46:29 2022] PUT /identity/v3/projects/56487a7b37614439a9245f4ebad97432/users/5ff5318ef7ae4f31885a53b79de2f3f5/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:29.593016 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9484c9ff-6778-48c6-86ef-b897c0d247ea None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:29.593268 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9484c9ff-6778-48c6-86ef-b897c0d247ea None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:29.593544 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9484c9ff-6778-48c6-86ef-b897c0d247ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:29.604294 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0558b3f-28fa-4314-8bb7-f3e907490def admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:29.630417 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-9484c9ff-6778-48c6-86ef-b897c0d247ea None None] MFA Rules not processed for user `5ff5318ef7ae4f31885a53b79de2f3f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:29.632053 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-9484c9ff-6778-48c6-86ef-b897c0d247ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:29.700382 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0558b3f-28fa-4314-8bb7-f3e907490def admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:29.704102 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ecd3bb-dc38-4128-b4dc-d82ff84497f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:29.705130 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ecd3bb-dc38-4128-b4dc-d82ff84497f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:29.705490 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ecd3bb-dc38-4128-b4dc-d82ff84497f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:29.706325 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ecd3bb-dc38-4128-b4dc-d82ff84497f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:29.708218 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-31ecd3bb-dc38-4128-b4dc-d82ff84497f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:29.755753 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 929/1855] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:29 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:29.761677 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9484c9ff-6778-48c6-86ef-b897c0d247ea None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:29.798630 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ecd3bb-dc38-4128-b4dc-d82ff84497f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:29.800347 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ecd3bb-dc38-4128-b4dc-d82ff84497f0 service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:29.812526 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 927/1856] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:46:29 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:46:29.855128 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9484c9ff-6778-48c6-86ef-b897c0d247ea None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:29.861345 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5764e06-0a2f-4bd8-b44b-94dd6b261f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:29.862770 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5764e06-0a2f-4bd8-b44b-94dd6b261f6d admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:29.863082 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5764e06-0a2f-4bd8-b44b-94dd6b261f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:29.863368 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5764e06-0a2f-4bd8-b44b-94dd6b261f6d admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:29.864150 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5764e06-0a2f-4bd8-b44b-94dd6b261f6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:29.869360 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5764e06-0a2f-4bd8-b44b-94dd6b261f6d admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:29.980931 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 930/1857] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:46:29 2022] POST /identity/v3/projects => generated 369 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:46:29.985191 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ecd3bb-dc38-4128-b4dc-d82ff84497f0 service cinder] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:29.993765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5764e06-0a2f-4bd8-b44b-94dd6b261f6d admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:29.995809 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-c5764e06-0a2f-4bd8-b44b-94dd6b261f6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:29.997235 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ecd3bb-dc38-4128-b4dc-d82ff84497f0 service cinder] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:30.063417 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834058f-3bc8-4c2e-a718-337f1aab9d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:30.064230 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834058f-3bc8-4c2e-a718-337f1aab9d3c admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:30.064513 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834058f-3bc8-4c2e-a718-337f1aab9d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:30.064826 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834058f-3bc8-4c2e-a718-337f1aab9d3c admin admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:30.065503 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2834058f-3bc8-4c2e-a718-337f1aab9d3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:30.067434 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2834058f-3bc8-4c2e-a718-337f1aab9d3c admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:30.077593 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5764e06-0a2f-4bd8-b44b-94dd6b261f6d admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:30.083055 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc32c0c-5e4d-49d4-aa8a-0635f291fc55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:30.083973 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc32c0c-5e4d-49d4-aa8a-0635f291fc55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:30.084263 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc32c0c-5e4d-49d4-aa8a-0635f291fc55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:30.084545 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc32c0c-5e4d-49d4-aa8a-0635f291fc55 admin admin] PATH_INFO: `/v3/projects/103740daa6b84bd08704436ea2688f20` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:30.085756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc32c0c-5e4d-49d4-aa8a-0635f291fc55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=103740daa6b84bd08704436ea2688f20)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:30.088941 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc32c0c-5e4d-49d4-aa8a-0635f291fc55 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:30.184738 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 928/1858] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:46:29 2022] POST /identity/v3/users => generated 291 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:46:30.186119 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834058f-3bc8-4c2e-a718-337f1aab9d3c admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:30.196765 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834058f-3bc8-4c2e-a718-337f1aab9d3c admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:30.201500 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0056de-b964-47e5-bc54-97944de43d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:30.202315 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0056de-b964-47e5-bc54-97944de43d3c admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:30.202424 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0056de-b964-47e5-bc54-97944de43d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:30.202706 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0056de-b964-47e5-bc54-97944de43d3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:30.203454 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0056de-b964-47e5-bc54-97944de43d3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:30.205202 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0056de-b964-47e5-bc54-97944de43d3c admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:30.222506 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 929/1859] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:46:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:46:30.228475 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0056de-b964-47e5-bc54-97944de43d3c admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:30.239286 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0056de-b964-47e5-bc54-97944de43d3c admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:30.244904 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-bcc32c0c-5e4d-49d4-aa8a-0635f291fc55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '103740daa6b84bd08704436ea2688f20', 'request_id': 'req-bcc32c0c-5e4d-49d4-aa8a-0635f291fc55'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:30.254149 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-bcc32c0c-5e4d-49d4-aa8a-0635f291fc55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '103740daa6b84bd08704436ea2688f20', 'request_id': 'req-bcc32c0c-5e4d-49d4-aa8a-0635f291fc55'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:30.261903 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-bcc32c0c-5e4d-49d4-aa8a-0635f291fc55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '103740daa6b84bd08704436ea2688f20', 'request_id': 'req-bcc32c0c-5e4d-49d4-aa8a-0635f291fc55'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:30.304935 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518fca5b-4049-44db-b25e-d63ef0fa2371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:30.305987 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518fca5b-4049-44db-b25e-d63ef0fa2371 admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:30.306272 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518fca5b-4049-44db-b25e-d63ef0fa2371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:30.306579 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518fca5b-4049-44db-b25e-d63ef0fa2371 admin admin] PATH_INFO: `/v3/projects/0bb1a49d249942f3836e14be47e99c04/users/fc1aed3f3028472791f53e60f2e8f05b/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:30.321517 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518fca5b-4049-44db-b25e-d63ef0fa2371 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bb1a49d249942f3836e14be47e99c04, user_id=fc1aed3f3028472791f53e60f2e8f05b, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:30.324225 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518fca5b-4049-44db-b25e-d63ef0fa2371 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:30.384792 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-bcc32c0c-5e4d-49d4-aa8a-0635f291fc55 admin admin] The token cache is being invalidate because project 103740daa6b84bd08704436ea2688f20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:46:30.385089 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-bcc32c0c-5e4d-49d4-aa8a-0635f291fc55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:30.385761 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-bcc32c0c-5e4d-49d4-aa8a-0635f291fc55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:30.387528 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 931/1860] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:46:29 2022] DELETE /identity/v3/projects/103740daa6b84bd08704436ea2688f20 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:46:30.391269 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc32c0c-5e4d-49d4-aa8a-0635f291fc55 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:30.392691 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-bcc32c0c-5e4d-49d4-aa8a-0635f291fc55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:30.406496 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc32c0c-5e4d-49d4-aa8a-0635f291fc55 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:30.411347 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b0ce3a-3361-4d55-80be-0c74cd196159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:30.412260 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b0ce3a-3361-4d55-80be-0c74cd196159 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:30.412544 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b0ce3a-3361-4d55-80be-0c74cd196159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:30.412989 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b0ce3a-3361-4d55-80be-0c74cd196159 admin admin] PATH_INFO: `/v3/users/2c54e3cc5cb241beb6b4e4d3e4fb1c51` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:30.414045 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b0ce3a-3361-4d55-80be-0c74cd196159 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c54e3cc5cb241beb6b4e4d3e4fb1c51)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:30.416499 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b0ce3a-3361-4d55-80be-0c74cd196159 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:30.448802 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 930/1861] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:46:30 2022] PUT /identity/v3/projects/0bb1a49d249942f3836e14be47e99c04/users/fc1aed3f3028472791f53e60f2e8f05b/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:46:30.456552 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518fca5b-4049-44db-b25e-d63ef0fa2371 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:30.458269 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-518fca5b-4049-44db-b25e-d63ef0fa2371 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:30.534808 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518fca5b-4049-44db-b25e-d63ef0fa2371 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:30.541164 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb443a69-87e6-450d-b37d-4e2196aa50d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:30.541901 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb443a69-87e6-450d-b37d-4e2196aa50d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:30.542186 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb443a69-87e6-450d-b37d-4e2196aa50d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:30.542462 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb443a69-87e6-450d-b37d-4e2196aa50d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:30.543198 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb443a69-87e6-450d-b37d-4e2196aa50d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:30.544914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb443a69-87e6-450d-b37d-4e2196aa50d4 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:30.556332 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 931/1862] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:46:30 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:46:30.562241 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb443a69-87e6-450d-b37d-4e2196aa50d4 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:30.575329 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb443a69-87e6-450d-b37d-4e2196aa50d4 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:30.582815 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd417717-24c2-418b-bd4a-568dff9163a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:30.585104 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd417717-24c2-418b-bd4a-568dff9163a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:30.585824 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd417717-24c2-418b-bd4a-568dff9163a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:30.587089 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd417717-24c2-418b-bd4a-568dff9163a9 admin admin] PATH_INFO: `/v3/projects/0bb1a49d249942f3836e14be47e99c04/users/fc1aed3f3028472791f53e60f2e8f05b/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:30.591152 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd417717-24c2-418b-bd4a-568dff9163a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bb1a49d249942f3836e14be47e99c04, user_id=fc1aed3f3028472791f53e60f2e8f05b, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:30.596830 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd417717-24c2-418b-bd4a-568dff9163a9 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:30.616686 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 932/1863] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:46:30 2022] PUT /identity/v3/projects/0bb1a49d249942f3836e14be47e99c04/users/fc1aed3f3028472791f53e60f2e8f05b/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:46:30.629320 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e6cf6-54a8-47eb-be9f-6078a5326374 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:30.630033 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e6cf6-54a8-47eb-be9f-6078a5326374 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:30.630340 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e6cf6-54a8-47eb-be9f-6078a5326374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:30.638790 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-75b0ce3a-3361-4d55-80be-0c74cd196159 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c54e3cc5cb241beb6b4e4d3e4fb1c51'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:30.669356 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-e98e6cf6-54a8-47eb-be9f-6078a5326374 None None] MFA Rules not processed for user `fc1aed3f3028472791f53e60f2e8f05b`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:30.670631 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-e98e6cf6-54a8-47eb-be9f-6078a5326374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:30.732182 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-75b0ce3a-3361-4d55-80be-0c74cd196159 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c54e3cc5cb241beb6b4e4d3e4fb1c51'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:30.745490 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-75b0ce3a-3361-4d55-80be-0c74cd196159 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c54e3cc5cb241beb6b4e4d3e4fb1c51'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:30.760109 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-75b0ce3a-3361-4d55-80be-0c74cd196159 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c54e3cc5cb241beb6b4e4d3e4fb1c51'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:30.760838 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-75b0ce3a-3361-4d55-80be-0c74cd196159 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c54e3cc5cb241beb6b4e4d3e4fb1c51'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:30.771336 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 932/1864] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:46:30 2022] DELETE /identity/v3/users/2c54e3cc5cb241beb6b4e4d3e4fb1c51 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:46:30.779386 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 933/1865] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:30 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:46:30.794297 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b0ce3a-3361-4d55-80be-0c74cd196159 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:30.796024 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-75b0ce3a-3361-4d55-80be-0c74cd196159 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:30.885809 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b0ce3a-3361-4d55-80be-0c74cd196159 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:30.889885 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017f9279-6008-473f-bb4f-97908e24b19c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:30.890559 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f9279-6008-473f-bb4f-97908e24b19c service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:30.890843 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f9279-6008-473f-bb4f-97908e24b19c service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:30.891133 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f9279-6008-473f-bb4f-97908e24b19c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:30.893562 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-017f9279-6008-473f-bb4f-97908e24b19c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:30.970284 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017f9279-6008-473f-bb4f-97908e24b19c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:30.972984 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017f9279-6008-473f-bb4f-97908e24b19c service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:30.989463 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 933/1866] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:46:30 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:46:31.004019 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:31.355408 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e6cf6-54a8-47eb-be9f-6078a5326374 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:31.459999 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e6cf6-54a8-47eb-be9f-6078a5326374 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:31.526966 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194a690-dc17-4bb0-a938-5edcce00e6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:31.528006 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194a690-dc17-4bb0-a938-5edcce00e6c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:31.528298 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194a690-dc17-4bb0-a938-5edcce00e6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:31.528578 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194a690-dc17-4bb0-a938-5edcce00e6c3 admin admin] PATH_INFO: `/v3/projects/281ade76a03b4ee48bd58dacce86c2ab` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:31.529815 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e194a690-dc17-4bb0-a938-5edcce00e6c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=281ade76a03b4ee48bd58dacce86c2ab)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:31.531752 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e194a690-dc17-4bb0-a938-5edcce00e6c3 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:31.673632 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-e194a690-dc17-4bb0-a938-5edcce00e6c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '281ade76a03b4ee48bd58dacce86c2ab', 'request_id': 'req-e194a690-dc17-4bb0-a938-5edcce00e6c3'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:31.680645 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-e194a690-dc17-4bb0-a938-5edcce00e6c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '281ade76a03b4ee48bd58dacce86c2ab', 'request_id': 'req-e194a690-dc17-4bb0-a938-5edcce00e6c3'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:31.778391 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-e194a690-dc17-4bb0-a938-5edcce00e6c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '281ade76a03b4ee48bd58dacce86c2ab', 'request_id': 'req-e194a690-dc17-4bb0-a938-5edcce00e6c3'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:31.793400 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-e194a690-dc17-4bb0-a938-5edcce00e6c3 admin admin] The token cache is being invalidate because project 281ade76a03b4ee48bd58dacce86c2ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:46:31.793852 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-e194a690-dc17-4bb0-a938-5edcce00e6c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:31.794935 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-e194a690-dc17-4bb0-a938-5edcce00e6c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:31.797094 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 934/1867] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:46:31 2022] DELETE /identity/v3/projects/281ade76a03b4ee48bd58dacce86c2ab => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:31.802111 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ab6c5-557a-4555-9d9f-286f10fa8875 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:31.802397 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ab6c5-557a-4555-9d9f-286f10fa8875 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:31.802707 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ab6c5-557a-4555-9d9f-286f10fa8875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:31.840235 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-9f2ab6c5-557a-4555-9d9f-286f10fa8875 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:31.841558 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-9f2ab6c5-557a-4555-9d9f-286f10fa8875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:31.906033 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 934/1868] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:31.915122 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70445595-eaf6-461e-8f75-d1e0e55d26a1 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:31.915409 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70445595-eaf6-461e-8f75-d1e0e55d26a1 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:31.915730 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70445595-eaf6-461e-8f75-d1e0e55d26a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:31.934434 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-70445595-eaf6-461e-8f75-d1e0e55d26a1 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:31.935703 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-70445595-eaf6-461e-8f75-d1e0e55d26a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:31.944203 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 935/1869] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:31.949384 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2ab6c5-557a-4555-9d9f-286f10fa8875 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:31.959485 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2ab6c5-557a-4555-9d9f-286f10fa8875 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:31.965097 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550a9651-8be2-42bf-8fec-d6cc211c9225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:31.965767 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a9651-8be2-42bf-8fec-d6cc211c9225 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:31.966055 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a9651-8be2-42bf-8fec-d6cc211c9225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:31.966331 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a9651-8be2-42bf-8fec-d6cc211c9225 admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:31.967369 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550a9651-8be2-42bf-8fec-d6cc211c9225 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:46:31.967508 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550a9651-8be2-42bf-8fec-d6cc211c9225 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:31.969129 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550a9651-8be2-42bf-8fec-d6cc211c9225 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:31.983571 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 935/1870] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 16:46:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:46:31.987090 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70445595-eaf6-461e-8f75-d1e0e55d26a1 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:31.996602 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70445595-eaf6-461e-8f75-d1e0e55d26a1 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:32.001574 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1002c22e-0692-4c87-a0ea-df5b2ec2a799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:32.002249 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002c22e-0692-4c87-a0ea-df5b2ec2a799 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:32.002523 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002c22e-0692-4c87-a0ea-df5b2ec2a799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:32.002802 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002c22e-0692-4c87-a0ea-df5b2ec2a799 admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:32.003526 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1002c22e-0692-4c87-a0ea-df5b2ec2a799 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:32.011346 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1002c22e-0692-4c87-a0ea-df5b2ec2a799 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:32.108201 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 936/1871] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:46:31 2022] POST /identity/v3/projects => generated 365 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:46:32.113230 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550a9651-8be2-42bf-8fec-d6cc211c9225 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:32.124823 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550a9651-8be2-42bf-8fec-d6cc211c9225 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:32.216362 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b8a2a-caa9-4446-9dfe-c97e27210ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:32.218530 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b8a2a-caa9-4446-9dfe-c97e27210ef1 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:32.219021 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b8a2a-caa9-4446-9dfe-c97e27210ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:32.219470 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b8a2a-caa9-4446-9dfe-c97e27210ef1 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:32.221315 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4b8a2a-caa9-4446-9dfe-c97e27210ef1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:32.224535 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4b8a2a-caa9-4446-9dfe-c97e27210ef1 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:32.332472 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1002c22e-0692-4c87-a0ea-df5b2ec2a799 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:32.332843 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 936/1872] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:46:32 2022] POST /identity/v3/users => generated 283 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:46:32.349607 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1002c22e-0692-4c87-a0ea-df5b2ec2a799 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:32.364803 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef62ffd-2c6a-4341-ae1d-6609f3eb15b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:32.365517 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef62ffd-2c6a-4341-ae1d-6609f3eb15b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:32.366151 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef62ffd-2c6a-4341-ae1d-6609f3eb15b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:32.366451 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef62ffd-2c6a-4341-ae1d-6609f3eb15b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:32.368403 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef62ffd-2c6a-4341-ae1d-6609f3eb15b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:32.372761 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef62ffd-2c6a-4341-ae1d-6609f3eb15b0 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:32.388711 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 937/1873] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:46:32 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:46:32.389824 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b8a2a-caa9-4446-9dfe-c97e27210ef1 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:32.403839 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b8a2a-caa9-4446-9dfe-c97e27210ef1 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:32.410436 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb49b536-1f83-4715-bbde-b6f557b4192b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:32.411332 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb49b536-1f83-4715-bbde-b6f557b4192b admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:32.411667 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb49b536-1f83-4715-bbde-b6f557b4192b admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:32.411952 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb49b536-1f83-4715-bbde-b6f557b4192b admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:32.412712 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb49b536-1f83-4715-bbde-b6f557b4192b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:32.414355 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb49b536-1f83-4715-bbde-b6f557b4192b admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:32.431658 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 937/1874] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:46:32 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:46:32.431912 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef62ffd-2c6a-4341-ae1d-6609f3eb15b0 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:32.442694 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef62ffd-2c6a-4341-ae1d-6609f3eb15b0 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:32.448921 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6ca8f8-e394-4298-b156-71110702a530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:32.449857 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6ca8f8-e394-4298-b156-71110702a530 admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:32.450142 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6ca8f8-e394-4298-b156-71110702a530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:32.450423 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6ca8f8-e394-4298-b156-71110702a530 admin admin] PATH_INFO: `/v3/projects/bc192e7e5a3b4a8da04f8f0b53105592/users/39607830620d46c08f574ee6bdbc5575/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:32.466704 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6ca8f8-e394-4298-b156-71110702a530 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc192e7e5a3b4a8da04f8f0b53105592, user_id=39607830620d46c08f574ee6bdbc5575, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:32.469577 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6ca8f8-e394-4298-b156-71110702a530 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:32.667552 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 938/1875] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:46:32 2022] PUT /identity/v3/projects/bc192e7e5a3b4a8da04f8f0b53105592/users/39607830620d46c08f574ee6bdbc5575/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:32.676544 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2889e238-fceb-4680-b1ff-76c90afcbcc1 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:32.676835 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2889e238-fceb-4680-b1ff-76c90afcbcc1 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:32.677113 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2889e238-fceb-4680-b1ff-76c90afcbcc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:32.709691 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-2889e238-fceb-4680-b1ff-76c90afcbcc1 None None] MFA Rules not processed for user `39607830620d46c08f574ee6bdbc5575`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:32.711375 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-2889e238-fceb-4680-b1ff-76c90afcbcc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:32.801578 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 938/1876] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:32 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:32.815708 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6ca8f8-e394-4298-b156-71110702a530 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:32.818185 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-1c6ca8f8-e394-4298-b156-71110702a530 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:32.937317 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6ca8f8-e394-4298-b156-71110702a530 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:32.946504 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacf1457-0c74-42ba-874c-0ec31b4e2f5f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0887908bcb0d4072bf6acb4cbaf1af83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:32.946504 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf1457-0c74-42ba-874c-0ec31b4e2f5f service glance] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:32.946504 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf1457-0c74-42ba-874c-0ec31b4e2f5f service glance] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:32.946504 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf1457-0c74-42ba-874c-0ec31b4e2f5f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:32.952936 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacf1457-0c74-42ba-874c-0ec31b4e2f5f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:32.954455 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacf1457-0c74-42ba-874c-0ec31b4e2f5f service glance] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:32.983667 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 939/1877] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:46:32 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:46:33.207793 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:35.036748 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:35.798242 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:37.124055 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2889e238-fceb-4680-b1ff-76c90afcbcc1 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:37.126225 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-2889e238-fceb-4680-b1ff-76c90afcbcc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:37.201490 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2889e238-fceb-4680-b1ff-76c90afcbcc1 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:37.207350 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88144be-9bdd-45c6-adea-6340393dfa56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:37.208276 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88144be-9bdd-45c6-adea-6340393dfa56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:37.208565 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88144be-9bdd-45c6-adea-6340393dfa56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:37.208849 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88144be-9bdd-45c6-adea-6340393dfa56 admin admin] PATH_INFO: `/v3/users/e93df74633a64940bb16b06aae63bfcc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:37.212515 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88144be-9bdd-45c6-adea-6340393dfa56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e93df74633a64940bb16b06aae63bfcc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:37.214444 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88144be-9bdd-45c6-adea-6340393dfa56 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:37.532036 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-a88144be-9bdd-45c6-adea-6340393dfa56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e93df74633a64940bb16b06aae63bfcc'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:37.609041 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-a88144be-9bdd-45c6-adea-6340393dfa56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e93df74633a64940bb16b06aae63bfcc'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:37.609438 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:37.620699 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-a88144be-9bdd-45c6-adea-6340393dfa56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e93df74633a64940bb16b06aae63bfcc'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:37.634771 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-a88144be-9bdd-45c6-adea-6340393dfa56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e93df74633a64940bb16b06aae63bfcc'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:37.638185 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-a88144be-9bdd-45c6-adea-6340393dfa56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e93df74633a64940bb16b06aae63bfcc'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:37.646427 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 939/1878] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:46:37 2022] DELETE /identity/v3/users/e93df74633a64940bb16b06aae63bfcc => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:37.652192 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ef786-a601-426c-b3b6-8dc1cbf6e851 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:37.652192 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ef786-a601-426c-b3b6-8dc1cbf6e851 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:37.652192 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ef786-a601-426c-b3b6-8dc1cbf6e851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:37.670068 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-ae2ef786-a601-426c-b3b6-8dc1cbf6e851 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:37.671329 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-ae2ef786-a601-426c-b3b6-8dc1cbf6e851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:37.747245 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 940/1879] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:46:37.759411 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88144be-9bdd-45c6-adea-6340393dfa56 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:37.761351 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a88144be-9bdd-45c6-adea-6340393dfa56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:37.828847 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88144be-9bdd-45c6-adea-6340393dfa56 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:37.834004 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab31e9eb-fa4e-47d5-b535-ba4cf1689537 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:37.834905 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab31e9eb-fa4e-47d5-b535-ba4cf1689537 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:37.835342 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab31e9eb-fa4e-47d5-b535-ba4cf1689537 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:37.835805 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab31e9eb-fa4e-47d5-b535-ba4cf1689537 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:37.847218 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab31e9eb-fa4e-47d5-b535-ba4cf1689537 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:37.848811 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab31e9eb-fa4e-47d5-b535-ba4cf1689537 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:37.859821 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 940/1880] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:46:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:46:37.918235 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:37.983318 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2ef786-a601-426c-b3b6-8dc1cbf6e851 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:37.984749 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-ae2ef786-a601-426c-b3b6-8dc1cbf6e851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:37.999071 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2ef786-a601-426c-b3b6-8dc1cbf6e851 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:38.003757 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379987e9-169f-41c4-b32f-e941053215eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:38.004653 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379987e9-169f-41c4-b32f-e941053215eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:38.008295 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379987e9-169f-41c4-b32f-e941053215eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:38.008295 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379987e9-169f-41c4-b32f-e941053215eb admin admin] PATH_INFO: `/v3/users/5ff5318ef7ae4f31885a53b79de2f3f5` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:38.008295 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379987e9-169f-41c4-b32f-e941053215eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ff5318ef7ae4f31885a53b79de2f3f5)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:38.009231 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379987e9-169f-41c4-b32f-e941053215eb admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:38.344560 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-379987e9-169f-41c4-b32f-e941053215eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ff5318ef7ae4f31885a53b79de2f3f5'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:38.457056 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab31e9eb-fa4e-47d5-b535-ba4cf1689537 service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:38.457056 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-ab31e9eb-fa4e-47d5-b535-ba4cf1689537 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:38.472925 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab31e9eb-fa4e-47d5-b535-ba4cf1689537 service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:38.482555 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ff03f2-ce1a-4a03-9500-41c91316545f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:38.483567 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff03f2-ce1a-4a03-9500-41c91316545f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:38.483995 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff03f2-ce1a-4a03-9500-41c91316545f admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:38.484137 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff03f2-ce1a-4a03-9500-41c91316545f admin admin] PATH_INFO: `/v3/projects/dab178dc67ed49cb90528aa8910acd52` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:38.484234 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-379987e9-169f-41c4-b32f-e941053215eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ff5318ef7ae4f31885a53b79de2f3f5'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:38.484860 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-379987e9-169f-41c4-b32f-e941053215eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ff5318ef7ae4f31885a53b79de2f3f5'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:38.485581 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-379987e9-169f-41c4-b32f-e941053215eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ff5318ef7ae4f31885a53b79de2f3f5'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:38.485664 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ff03f2-ce1a-4a03-9500-41c91316545f admin admin] RBAC: Authorizing `identity:delete_project(project_id=dab178dc67ed49cb90528aa8910acd52)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:38.491981 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ff03f2-ce1a-4a03-9500-41c91316545f admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:38.493718 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-379987e9-169f-41c4-b32f-e941053215eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ff5318ef7ae4f31885a53b79de2f3f5'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:38.517879 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 941/1881] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:46:37 2022] DELETE /identity/v3/users/5ff5318ef7ae4f31885a53b79de2f3f5 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:38.522285 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a87164-0521-4281-a819-401c8caa0c65 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:38.522585 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a87164-0521-4281-a819-401c8caa0c65 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:38.522870 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a87164-0521-4281-a819-401c8caa0c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:38.544036 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-96a87164-0521-4281-a819-401c8caa0c65 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:38.545299 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-96a87164-0521-4281-a819-401c8caa0c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:38.605420 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-32ff03f2-ce1a-4a03-9500-41c91316545f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dab178dc67ed49cb90528aa8910acd52', 'request_id': 'req-32ff03f2-ce1a-4a03-9500-41c91316545f'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:38.611908 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-32ff03f2-ce1a-4a03-9500-41c91316545f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dab178dc67ed49cb90528aa8910acd52', 'request_id': 'req-32ff03f2-ce1a-4a03-9500-41c91316545f'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:38.614338 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 942/1882] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:46:38.622684 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-32ff03f2-ce1a-4a03-9500-41c91316545f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dab178dc67ed49cb90528aa8910acd52', 'request_id': 'req-32ff03f2-ce1a-4a03-9500-41c91316545f'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:38.624791 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a87164-0521-4281-a819-401c8caa0c65 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:38.627339 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-96a87164-0521-4281-a819-401c8caa0c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:38.684620 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a87164-0521-4281-a819-401c8caa0c65 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:38.735001 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f937c07-ca3d-4e30-a7db-cdbad41360be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:38.735698 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f937c07-ca3d-4e30-a7db-cdbad41360be service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:38.735993 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f937c07-ca3d-4e30-a7db-cdbad41360be service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:38.736525 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f937c07-ca3d-4e30-a7db-cdbad41360be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:38.740183 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-32ff03f2-ce1a-4a03-9500-41c91316545f admin admin] The token cache is being invalidate because project dab178dc67ed49cb90528aa8910acd52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:46:38.742020 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-32ff03f2-ce1a-4a03-9500-41c91316545f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:38.742020 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-32ff03f2-ce1a-4a03-9500-41c91316545f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:38.742941 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 941/1883] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:46:38 2022] DELETE /identity/v3/projects/dab178dc67ed49cb90528aa8910acd52 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:38.746800 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ff03f2-ce1a-4a03-9500-41c91316545f admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:38.748227 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-32ff03f2-ce1a-4a03-9500-41c91316545f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:38.801069 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f937c07-ca3d-4e30-a7db-cdbad41360be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:38.802578 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f937c07-ca3d-4e30-a7db-cdbad41360be service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:38.813401 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 943/1884] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:46:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:46:38.816249 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ff03f2-ce1a-4a03-9500-41c91316545f admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:38.822281 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5a7c0d-f855-4341-9f13-b71f9f00a4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:38.823351 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5a7c0d-f855-4341-9f13-b71f9f00a4bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:38.823667 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5a7c0d-f855-4341-9f13-b71f9f00a4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:38.823957 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5a7c0d-f855-4341-9f13-b71f9f00a4bf admin admin] PATH_INFO: `/v3/users/f3194adc8f764b11ba06b1ec41eace43` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:38.825207 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5a7c0d-f855-4341-9f13-b71f9f00a4bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3194adc8f764b11ba06b1ec41eace43)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:38.833223 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5a7c0d-f855-4341-9f13-b71f9f00a4bf admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:39.011197 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:39.084643 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-8c5a7c0d-f855-4341-9f13-b71f9f00a4bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3194adc8f764b11ba06b1ec41eace43'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:39.151860 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-8c5a7c0d-f855-4341-9f13-b71f9f00a4bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3194adc8f764b11ba06b1ec41eace43'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:39.159449 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-8c5a7c0d-f855-4341-9f13-b71f9f00a4bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3194adc8f764b11ba06b1ec41eace43'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:39.171908 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-8c5a7c0d-f855-4341-9f13-b71f9f00a4bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3194adc8f764b11ba06b1ec41eace43'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:39.172641 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-8c5a7c0d-f855-4341-9f13-b71f9f00a4bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3194adc8f764b11ba06b1ec41eace43'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:39.178856 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 942/1885] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:46:38 2022] DELETE /identity/v3/users/f3194adc8f764b11ba06b1ec41eace43 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:46:39.335524 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f937c07-ca3d-4e30-a7db-cdbad41360be service neutron] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:39.337408 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-7f937c07-ca3d-4e30-a7db-cdbad41360be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:39.363404 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f937c07-ca3d-4e30-a7db-cdbad41360be service neutron] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:39.450842 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10c77b0-ea74-410c-8a3a-4d1ea1f41475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:39.453979 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c77b0-ea74-410c-8a3a-4d1ea1f41475 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:39.453979 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c77b0-ea74-410c-8a3a-4d1ea1f41475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:39.453979 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c77b0-ea74-410c-8a3a-4d1ea1f41475 admin admin] PATH_INFO: `/v3/projects/56487a7b37614439a9245f4ebad97432` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:39.453979 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10c77b0-ea74-410c-8a3a-4d1ea1f41475 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56487a7b37614439a9245f4ebad97432)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:39.455898 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10c77b0-ea74-410c-8a3a-4d1ea1f41475 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:39.600983 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-e10c77b0-ea74-410c-8a3a-4d1ea1f41475 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56487a7b37614439a9245f4ebad97432', 'request_id': 'req-e10c77b0-ea74-410c-8a3a-4d1ea1f41475'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:39.608758 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-e10c77b0-ea74-410c-8a3a-4d1ea1f41475 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56487a7b37614439a9245f4ebad97432', 'request_id': 'req-e10c77b0-ea74-410c-8a3a-4d1ea1f41475'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:39.702198 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:39.703406 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-e10c77b0-ea74-410c-8a3a-4d1ea1f41475 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56487a7b37614439a9245f4ebad97432', 'request_id': 'req-e10c77b0-ea74-410c-8a3a-4d1ea1f41475'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:39.715975 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-e10c77b0-ea74-410c-8a3a-4d1ea1f41475 admin admin] The token cache is being invalidate because project 56487a7b37614439a9245f4ebad97432 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:46:39.716272 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-e10c77b0-ea74-410c-8a3a-4d1ea1f41475 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:39.716960 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-e10c77b0-ea74-410c-8a3a-4d1ea1f41475 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:39.718554 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 944/1886] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:46:39 2022] DELETE /identity/v3/projects/56487a7b37614439a9245f4ebad97432 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:39.722539 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5a7c0d-f855-4341-9f13-b71f9f00a4bf admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:39.724061 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-8c5a7c0d-f855-4341-9f13-b71f9f00a4bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:39.738279 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5a7c0d-f855-4341-9f13-b71f9f00a4bf admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:39.794776 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10c77b0-ea74-410c-8a3a-4d1ea1f41475 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:39.796299 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-e10c77b0-ea74-410c-8a3a-4d1ea1f41475 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:39.803722 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c14b97-5862-49d0-b259-39e8b6ab7c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:39.803722 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c14b97-5862-49d0-b259-39e8b6ab7c96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:39.803722 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c14b97-5862-49d0-b259-39e8b6ab7c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:39.803722 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c14b97-5862-49d0-b259-39e8b6ab7c96 admin admin] PATH_INFO: `/v3/users/fc1aed3f3028472791f53e60f2e8f05b` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:39.806036 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c14b97-5862-49d0-b259-39e8b6ab7c96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc1aed3f3028472791f53e60f2e8f05b)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:39.807987 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c14b97-5862-49d0-b259-39e8b6ab7c96 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:39.814763 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10c77b0-ea74-410c-8a3a-4d1ea1f41475 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:39.819661 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb6702f-f899-4e6a-95eb-070ed6185bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:39.820866 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb6702f-f899-4e6a-95eb-070ed6185bdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:39.821694 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb6702f-f899-4e6a-95eb-070ed6185bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:39.822017 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb6702f-f899-4e6a-95eb-070ed6185bdb admin admin] PATH_INFO: `/v3/projects/f45ab42b014b4523b145fe60a38b467f` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:39.823346 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb6702f-f899-4e6a-95eb-070ed6185bdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=f45ab42b014b4523b145fe60a38b467f)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:39.825371 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb6702f-f899-4e6a-95eb-070ed6185bdb admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:40.057660 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-adb6702f-f899-4e6a-95eb-070ed6185bdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f45ab42b014b4523b145fe60a38b467f', 'request_id': 'req-adb6702f-f899-4e6a-95eb-070ed6185bdb'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:40.058205 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-e7c14b97-5862-49d0-b259-39e8b6ab7c96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc1aed3f3028472791f53e60f2e8f05b'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:40.065172 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-adb6702f-f899-4e6a-95eb-070ed6185bdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f45ab42b014b4523b145fe60a38b467f', 'request_id': 'req-adb6702f-f899-4e6a-95eb-070ed6185bdb'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:40.138938 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-e7c14b97-5862-49d0-b259-39e8b6ab7c96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc1aed3f3028472791f53e60f2e8f05b'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:40.147442 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-e7c14b97-5862-49d0-b259-39e8b6ab7c96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc1aed3f3028472791f53e60f2e8f05b'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:40.162271 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-e7c14b97-5862-49d0-b259-39e8b6ab7c96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc1aed3f3028472791f53e60f2e8f05b'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:40.163191 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-e7c14b97-5862-49d0-b259-39e8b6ab7c96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc1aed3f3028472791f53e60f2e8f05b'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:40.170495 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 943/1887] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:46:39 2022] DELETE /identity/v3/users/fc1aed3f3028472791f53e60f2e8f05b => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:40.172031 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:40.214382 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-adb6702f-f899-4e6a-95eb-070ed6185bdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f45ab42b014b4523b145fe60a38b467f', 'request_id': 'req-adb6702f-f899-4e6a-95eb-070ed6185bdb'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:40.227715 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-adb6702f-f899-4e6a-95eb-070ed6185bdb admin admin] The token cache is being invalidate because project f45ab42b014b4523b145fe60a38b467f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:46:40.228152 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-adb6702f-f899-4e6a-95eb-070ed6185bdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:40.228840 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-adb6702f-f899-4e6a-95eb-070ed6185bdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:40.230691 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 945/1888] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:46:39 2022] DELETE /identity/v3/projects/f45ab42b014b4523b145fe60a38b467f => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:40.238215 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f614d650-6cb1-450f-95f3-96cd91c362c1 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:40.238946 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f614d650-6cb1-450f-95f3-96cd91c362c1 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:40.239232 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f614d650-6cb1-450f-95f3-96cd91c362c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:40.261860 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-f614d650-6cb1-450f-95f3-96cd91c362c1 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:40.263153 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f614d650-6cb1-450f-95f3-96cd91c362c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:40.329978 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 944/1889] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:46:40.339045 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cecac2-223c-48ae-b16b-6755c45c3d7a None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:40.344121 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cecac2-223c-48ae-b16b-6755c45c3d7a None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:40.344121 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cecac2-223c-48ae-b16b-6755c45c3d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:40.360262 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-57cecac2-223c-48ae-b16b-6755c45c3d7a None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:40.361513 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-57cecac2-223c-48ae-b16b-6755c45c3d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:40.371248 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 946/1890] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:40.375519 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f614d650-6cb1-450f-95f3-96cd91c362c1 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:40.387518 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f614d650-6cb1-450f-95f3-96cd91c362c1 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:40.392520 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8aa262-194a-46fe-8f3d-90c415cc11af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:40.393354 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8aa262-194a-46fe-8f3d-90c415cc11af admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:40.393642 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8aa262-194a-46fe-8f3d-90c415cc11af admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:40.395698 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8aa262-194a-46fe-8f3d-90c415cc11af admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:40.395698 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8aa262-194a-46fe-8f3d-90c415cc11af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:46:40.395698 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8aa262-194a-46fe-8f3d-90c415cc11af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:40.396789 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8aa262-194a-46fe-8f3d-90c415cc11af admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:40.414190 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 945/1891] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 16:46:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:46:40.418340 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cecac2-223c-48ae-b16b-6755c45c3d7a None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:40.434444 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cecac2-223c-48ae-b16b-6755c45c3d7a None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:40.439539 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e3d4fa-7680-4b1b-ba81-2eab52bd9447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:40.440219 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e3d4fa-7680-4b1b-ba81-2eab52bd9447 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:40.440504 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e3d4fa-7680-4b1b-ba81-2eab52bd9447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:40.440785 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e3d4fa-7680-4b1b-ba81-2eab52bd9447 admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:40.441497 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e3d4fa-7680-4b1b-ba81-2eab52bd9447 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:40.443517 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e3d4fa-7680-4b1b-ba81-2eab52bd9447 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:40.536486 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 947/1892] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:46:40 2022] POST /identity/v3/projects => generated 385 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:46:40.540840 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8aa262-194a-46fe-8f3d-90c415cc11af admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:40.551530 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8aa262-194a-46fe-8f3d-90c415cc11af admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:40.623793 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70860948-8bfd-474b-aa44-5db1a55e5738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:40.624484 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70860948-8bfd-474b-aa44-5db1a55e5738 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:40.625116 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70860948-8bfd-474b-aa44-5db1a55e5738 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:40.625408 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70860948-8bfd-474b-aa44-5db1a55e5738 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:40.626110 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70860948-8bfd-474b-aa44-5db1a55e5738 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:40.628268 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70860948-8bfd-474b-aa44-5db1a55e5738 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:40.696366 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e3d4fa-7680-4b1b-ba81-2eab52bd9447 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:40.700434 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-35e3d4fa-7680-4b1b-ba81-2eab52bd9447 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:40.745959 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e3d4fa-7680-4b1b-ba81-2eab52bd9447 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:40.753980 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035d3227-361b-4a9e-8972-77f3e1866a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:40.754986 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d3227-361b-4a9e-8972-77f3e1866a15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:40.755425 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d3227-361b-4a9e-8972-77f3e1866a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:40.756083 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d3227-361b-4a9e-8972-77f3e1866a15 admin admin] PATH_INFO: `/v3/projects/0bb1a49d249942f3836e14be47e99c04` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:40.766283 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035d3227-361b-4a9e-8972-77f3e1866a15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bb1a49d249942f3836e14be47e99c04)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:40.770603 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035d3227-361b-4a9e-8972-77f3e1866a15 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:40.786672 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 946/1893] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:46:40 2022] POST /identity/v3/users => generated 293 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:46:40.788704 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70860948-8bfd-474b-aa44-5db1a55e5738 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:40.804406 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70860948-8bfd-474b-aa44-5db1a55e5738 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:40.812542 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b87872-7a2d-4e3c-9b30-c4efbf493f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:40.813855 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b87872-7a2d-4e3c-9b30-c4efbf493f9a admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:40.814437 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b87872-7a2d-4e3c-9b30-c4efbf493f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:40.815025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b87872-7a2d-4e3c-9b30-c4efbf493f9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:40.816604 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b87872-7a2d-4e3c-9b30-c4efbf493f9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:40.818420 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b87872-7a2d-4e3c-9b30-c4efbf493f9a admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:40.842238 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 947/1894] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:46:40 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:46:40.843616 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b87872-7a2d-4e3c-9b30-c4efbf493f9a admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:40.859805 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b87872-7a2d-4e3c-9b30-c4efbf493f9a admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:40.865719 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ffa64a-e5d4-4661-b67e-27e3994c79bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:40.866405 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ffa64a-e5d4-4661-b67e-27e3994c79bf admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:40.866689 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ffa64a-e5d4-4661-b67e-27e3994c79bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:40.867125 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ffa64a-e5d4-4661-b67e-27e3994c79bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:40.868465 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ffa64a-e5d4-4661-b67e-27e3994c79bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:40.871070 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ffa64a-e5d4-4661-b67e-27e3994c79bf admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:40.886025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 948/1895] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:46:40 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:46:40.888059 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ffa64a-e5d4-4661-b67e-27e3994c79bf admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:40.898926 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ffa64a-e5d4-4661-b67e-27e3994c79bf admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:40.916978 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-035d3227-361b-4a9e-8972-77f3e1866a15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bb1a49d249942f3836e14be47e99c04', 'request_id': 'req-035d3227-361b-4a9e-8972-77f3e1866a15'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:40.924270 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-035d3227-361b-4a9e-8972-77f3e1866a15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bb1a49d249942f3836e14be47e99c04', 'request_id': 'req-035d3227-361b-4a9e-8972-77f3e1866a15'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:40.967963 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cdbafb-f5e1-40fd-a4a3-7cebc3c8571b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:40.969024 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cdbafb-f5e1-40fd-a4a3-7cebc3c8571b admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:40.969305 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cdbafb-f5e1-40fd-a4a3-7cebc3c8571b admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:40.969580 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cdbafb-f5e1-40fd-a4a3-7cebc3c8571b admin admin] PATH_INFO: `/v3/projects/fbf459f524444ce8b3ebc7ca37f80468/users/7f4e4df43d9d48f481d8be85380f5a58/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:40.986189 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cdbafb-f5e1-40fd-a4a3-7cebc3c8571b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbf459f524444ce8b3ebc7ca37f80468, user_id=7f4e4df43d9d48f481d8be85380f5a58, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:40.989012 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cdbafb-f5e1-40fd-a4a3-7cebc3c8571b admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:40.998352 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-035d3227-361b-4a9e-8972-77f3e1866a15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bb1a49d249942f3836e14be47e99c04', 'request_id': 'req-035d3227-361b-4a9e-8972-77f3e1866a15'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:41.009948 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-035d3227-361b-4a9e-8972-77f3e1866a15 admin admin] The token cache is being invalidate because project 0bb1a49d249942f3836e14be47e99c04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:46:41.010239 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-035d3227-361b-4a9e-8972-77f3e1866a15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:41.010948 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-035d3227-361b-4a9e-8972-77f3e1866a15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:41.012587 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 948/1896] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:46:40 2022] DELETE /identity/v3/projects/0bb1a49d249942f3836e14be47e99c04 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:46:41.017231 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b475be-b2df-4815-b658-979c4ebc067b None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:41.017536 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b475be-b2df-4815-b658-979c4ebc067b None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:41.017820 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b475be-b2df-4815-b658-979c4ebc067b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:41.036581 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-66b475be-b2df-4815-b658-979c4ebc067b None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:41.037807 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-66b475be-b2df-4815-b658-979c4ebc067b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:41.047230 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 949/1897] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:46:41.055966 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced71749-0d6d-459c-95a3-406f8844e063 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:41.056290 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced71749-0d6d-459c-95a3-406f8844e063 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:41.056760 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced71749-0d6d-459c-95a3-406f8844e063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:41.076302 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-ced71749-0d6d-459c-95a3-406f8844e063 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:41.077522 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-ced71749-0d6d-459c-95a3-406f8844e063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:41.086135 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 950/1898] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:46:41.091455 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced71749-0d6d-459c-95a3-406f8844e063 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:41.101529 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced71749-0d6d-459c-95a3-406f8844e063 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:41.103508 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 949/1899] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:46:40 2022] PUT /identity/v3/projects/fbf459f524444ce8b3ebc7ca37f80468/users/7f4e4df43d9d48f481d8be85380f5a58/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:46:41.114008 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf7d34-5be8-437a-8af1-78d79ce24551 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:41.114553 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf7d34-5be8-437a-8af1-78d79ce24551 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:41.114947 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf7d34-5be8-437a-8af1-78d79ce24551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:41.144114 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-41bf7d34-5be8-437a-8af1-78d79ce24551 None None] MFA Rules not processed for user `7f4e4df43d9d48f481d8be85380f5a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:41.145437 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-41bf7d34-5be8-437a-8af1-78d79ce24551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:41.187589 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc848e48-9746-4445-b6f0-2e19afd877c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:41.189227 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc848e48-9746-4445-b6f0-2e19afd877c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:41.189705 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc848e48-9746-4445-b6f0-2e19afd877c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:41.189820 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc848e48-9746-4445-b6f0-2e19afd877c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:41.191068 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc848e48-9746-4445-b6f0-2e19afd877c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:46:41.191068 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc848e48-9746-4445-b6f0-2e19afd877c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:41.192491 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc848e48-9746-4445-b6f0-2e19afd877c2 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:41.219573 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 951/1900] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 16:46:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:46:41.223836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc848e48-9746-4445-b6f0-2e19afd877c2 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:41.235797 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc848e48-9746-4445-b6f0-2e19afd877c2 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:41.243528 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e20ba1-47f1-4c82-a6cf-f866b1549c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:41.244425 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e20ba1-47f1-4c82-a6cf-f866b1549c1a admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:41.244716 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e20ba1-47f1-4c82-a6cf-f866b1549c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:41.244996 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e20ba1-47f1-4c82-a6cf-f866b1549c1a admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:41.245700 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e20ba1-47f1-4c82-a6cf-f866b1549c1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:41.248651 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 950/1901] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:46:41.248782 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e20ba1-47f1-4c82-a6cf-f866b1549c1a admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:41.264044 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bf7d34-5be8-437a-8af1-78d79ce24551 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:41.265804 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-41bf7d34-5be8-437a-8af1-78d79ce24551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:41.328397 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bf7d34-5be8-437a-8af1-78d79ce24551 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:41.332082 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d88e000-aba1-4e41-a604-367b08f74d2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:41.332739 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88e000-aba1-4e41-a604-367b08f74d2d service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:41.333032 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88e000-aba1-4e41-a604-367b08f74d2d service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:41.333441 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88e000-aba1-4e41-a604-367b08f74d2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:41.343385 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d88e000-aba1-4e41-a604-367b08f74d2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:41.343596 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 952/1902] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:46:41 2022] POST /identity/v3/projects => generated 385 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:46:41.344809 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d88e000-aba1-4e41-a604-367b08f74d2d service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:41.350312 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e20ba1-47f1-4c82-a6cf-f866b1549c1a admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:41.356567 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 951/1903] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:46:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:46:41.424972 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e20ba1-47f1-4c82-a6cf-f866b1549c1a admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:41.429800 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04d50d-16c9-434a-80b9-f88b2c72cfe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:41.430795 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04d50d-16c9-434a-80b9-f88b2c72cfe9 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:41.431361 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04d50d-16c9-434a-80b9-f88b2c72cfe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:41.431815 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04d50d-16c9-434a-80b9-f88b2c72cfe9 admin admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:41.433018 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d04d50d-16c9-434a-80b9-f88b2c72cfe9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:41.435560 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d04d50d-16c9-434a-80b9-f88b2c72cfe9 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:41.510578 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 953/1904] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:46:41 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:46:41.514845 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d88e000-aba1-4e41-a604-367b08f74d2d service cinder] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:41.526364 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d88e000-aba1-4e41-a604-367b08f74d2d service cinder] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:41.531024 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4e626d-8b14-456b-9f34-f087d2ee4750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:41.531710 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e626d-8b14-456b-9f34-f087d2ee4750 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:41.531996 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e626d-8b14-456b-9f34-f087d2ee4750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:41.532272 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e626d-8b14-456b-9f34-f087d2ee4750 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:41.532993 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4e626d-8b14-456b-9f34-f087d2ee4750 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:41.534600 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4e626d-8b14-456b-9f34-f087d2ee4750 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:41.545705 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 952/1905] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:46:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:46:41.550032 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04d50d-16c9-434a-80b9-f88b2c72cfe9 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:41.560701 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04d50d-16c9-434a-80b9-f88b2c72cfe9 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:41.565641 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00c930c-7542-4749-b259-ea174403405c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:41.566328 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c930c-7542-4749-b259-ea174403405c admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:41.566604 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c930c-7542-4749-b259-ea174403405c admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:41.566878 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c930c-7542-4749-b259-ea174403405c admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:41.567670 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00c930c-7542-4749-b259-ea174403405c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:41.569284 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00c930c-7542-4749-b259-ea174403405c admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:41.580760 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 954/1906] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:46:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:46:41.585085 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4e626d-8b14-456b-9f34-f087d2ee4750 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:41.596779 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4e626d-8b14-456b-9f34-f087d2ee4750 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:41.601506 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe8dd3b-2ce5-4463-b015-a651c472e752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:41.604319 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe8dd3b-2ce5-4463-b015-a651c472e752 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:41.604655 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe8dd3b-2ce5-4463-b015-a651c472e752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:41.604937 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe8dd3b-2ce5-4463-b015-a651c472e752 admin admin] PATH_INFO: `/v3/projects/505b81605c2e4202901ea9b0480512e8/users/6ac6b0957d4c43589a1e9289a9c71117/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:41.622409 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe8dd3b-2ce5-4463-b015-a651c472e752 admin admin] RBAC: Authorizing `identity:create_grant(project_id=505b81605c2e4202901ea9b0480512e8, user_id=6ac6b0957d4c43589a1e9289a9c71117, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:41.625111 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe8dd3b-2ce5-4463-b015-a651c472e752 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:41.712758 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 953/1907] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:46:41 2022] PUT /identity/v3/projects/505b81605c2e4202901ea9b0480512e8/users/6ac6b0957d4c43589a1e9289a9c71117/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:41.724943 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1b872-012b-4b42-a842-e1ab40ab6f25 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:41.725323 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1b872-012b-4b42-a842-e1ab40ab6f25 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:41.725808 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1b872-012b-4b42-a842-e1ab40ab6f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:41.756747 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-cda1b872-012b-4b42-a842-e1ab40ab6f25 None None] MFA Rules not processed for user `6ac6b0957d4c43589a1e9289a9c71117`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:41.758007 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-cda1b872-012b-4b42-a842-e1ab40ab6f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:41.829272 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 955/1908] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:41.833771 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe8dd3b-2ce5-4463-b015-a651c472e752 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:41.856142 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:41.907057 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe8dd3b-2ce5-4463-b015-a651c472e752 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:41.918697 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a033687-edd5-4d21-baa9-a7e77987db97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:41.919389 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a033687-edd5-4d21-baa9-a7e77987db97 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:41.919699 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a033687-edd5-4d21-baa9-a7e77987db97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:41.919976 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a033687-edd5-4d21-baa9-a7e77987db97 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:41.920698 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a033687-edd5-4d21-baa9-a7e77987db97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:41.922813 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a033687-edd5-4d21-baa9-a7e77987db97 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:42.035084 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 954/1909] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:46:41 2022] POST /identity/v3/projects => generated 387 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:46:42.039120 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda1b872-012b-4b42-a842-e1ab40ab6f25 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:42.054915 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda1b872-012b-4b42-a842-e1ab40ab6f25 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:42.140583 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de5d132-e0d4-4ebd-bbe6-12844ca6e782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:42.141404 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5d132-e0d4-4ebd-bbe6-12844ca6e782 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:42.141698 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5d132-e0d4-4ebd-bbe6-12844ca6e782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:42.141977 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5d132-e0d4-4ebd-bbe6-12844ca6e782 admin admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:42.142712 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de5d132-e0d4-4ebd-bbe6-12844ca6e782 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:42.144738 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de5d132-e0d4-4ebd-bbe6-12844ca6e782 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:42.166184 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:42.260104 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 956/1910] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:46:42 2022] POST /identity/v3/users => generated 294 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:46:42.261382 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a033687-edd5-4d21-baa9-a7e77987db97 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:42.273031 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a033687-edd5-4d21-baa9-a7e77987db97 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:42.279531 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ac5fb3-7f2b-4ac9-9f32-17435f56d4cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:42.280393 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac5fb3-7f2b-4ac9-9f32-17435f56d4cb admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:42.280745 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac5fb3-7f2b-4ac9-9f32-17435f56d4cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:42.281095 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac5fb3-7f2b-4ac9-9f32-17435f56d4cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:42.282070 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ac5fb3-7f2b-4ac9-9f32-17435f56d4cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:42.283815 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ac5fb3-7f2b-4ac9-9f32-17435f56d4cb admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:42.299540 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 955/1911] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:46:42 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:46:42.302674 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de5d132-e0d4-4ebd-bbe6-12844ca6e782 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:42.315137 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de5d132-e0d4-4ebd-bbe6-12844ca6e782 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:42.320147 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48df9dc-5e91-4151-923c-a6d03868c690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:42.321603 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48df9dc-5e91-4151-923c-a6d03868c690 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:42.321885 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48df9dc-5e91-4151-923c-a6d03868c690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:42.322263 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48df9dc-5e91-4151-923c-a6d03868c690 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:42.323006 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48df9dc-5e91-4151-923c-a6d03868c690 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:42.324649 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48df9dc-5e91-4151-923c-a6d03868c690 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:42.335324 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 957/1912] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:46:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:46:42.340826 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ac5fb3-7f2b-4ac9-9f32-17435f56d4cb admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:42.351171 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ac5fb3-7f2b-4ac9-9f32-17435f56d4cb admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:42.356303 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50c912b-c6b3-4e8f-85fa-7469c46b8564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:42.357174 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c912b-c6b3-4e8f-85fa-7469c46b8564 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:42.358233 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c912b-c6b3-4e8f-85fa-7469c46b8564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:42.358233 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c912b-c6b3-4e8f-85fa-7469c46b8564 admin admin] PATH_INFO: `/v3/projects/8462c49b73fd464ba9abc47ada56d21b/users/13c6426c31f44abdb9261d41d5b9cb9c/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:42.374034 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50c912b-c6b3-4e8f-85fa-7469c46b8564 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8462c49b73fd464ba9abc47ada56d21b, user_id=13c6426c31f44abdb9261d41d5b9cb9c, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:42.376755 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50c912b-c6b3-4e8f-85fa-7469c46b8564 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:42.450363 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 956/1913] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:46:42 2022] PUT /identity/v3/projects/8462c49b73fd464ba9abc47ada56d21b/users/13c6426c31f44abdb9261d41d5b9cb9c/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:42.462767 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de68923-7e17-47ec-82a0-0e01198c9e82 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:42.463010 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de68923-7e17-47ec-82a0-0e01198c9e82 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:42.463295 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de68923-7e17-47ec-82a0-0e01198c9e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:42.490452 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-5de68923-7e17-47ec-82a0-0e01198c9e82 None None] MFA Rules not processed for user `13c6426c31f44abdb9261d41d5b9cb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:42.491742 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-5de68923-7e17-47ec-82a0-0e01198c9e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:42.557708 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 958/1914] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:42.562456 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50c912b-c6b3-4e8f-85fa-7469c46b8564 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:42.626760 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50c912b-c6b3-4e8f-85fa-7469c46b8564 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:42.631482 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d056d226-4ba0-4489-984d-ee3ca0fe1362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:42.632055 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d056d226-4ba0-4489-984d-ee3ca0fe1362 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:42.632339 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d056d226-4ba0-4489-984d-ee3ca0fe1362 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:42.632618 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d056d226-4ba0-4489-984d-ee3ca0fe1362 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:42.633817 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d056d226-4ba0-4489-984d-ee3ca0fe1362 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:42.635819 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d056d226-4ba0-4489-984d-ee3ca0fe1362 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:42.727640 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 957/1915] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:46:42 2022] POST /identity/v3/projects => generated 385 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:46:42.728650 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de68923-7e17-47ec-82a0-0e01198c9e82 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:42.740367 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de68923-7e17-47ec-82a0-0e01198c9e82 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:42.805873 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c3db6f-92e0-4cdd-9a27-c4260536b2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:42.806689 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3db6f-92e0-4cdd-9a27-c4260536b2cc admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:42.806828 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3db6f-92e0-4cdd-9a27-c4260536b2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:42.807227 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3db6f-92e0-4cdd-9a27-c4260536b2cc admin admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:42.807975 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c3db6f-92e0-4cdd-9a27-c4260536b2cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:42.809897 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c3db6f-92e0-4cdd-9a27-c4260536b2cc admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:42.943428 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 959/1916] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:46:42 2022] POST /identity/v3/users => generated 299 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:46:42.948390 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d056d226-4ba0-4489-984d-ee3ca0fe1362 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:42.959597 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d056d226-4ba0-4489-984d-ee3ca0fe1362 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:42.966731 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f8c0c4-9081-49d2-8729-1e415282eba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:42.967445 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f8c0c4-9081-49d2-8729-1e415282eba2 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:42.968205 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f8c0c4-9081-49d2-8729-1e415282eba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:42.968205 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f8c0c4-9081-49d2-8729-1e415282eba2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:42.968795 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f8c0c4-9081-49d2-8729-1e415282eba2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:42.970441 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f8c0c4-9081-49d2-8729-1e415282eba2 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:42.982271 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 958/1917] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:46:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:46:42.986768 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c3db6f-92e0-4cdd-9a27-c4260536b2cc admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:42.998555 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c3db6f-92e0-4cdd-9a27-c4260536b2cc admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:43.003675 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3050466-af0e-4fd0-b670-ec6aac049086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:43.010838 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3050466-af0e-4fd0-b670-ec6aac049086 admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:43.010838 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3050466-af0e-4fd0-b670-ec6aac049086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:43.010838 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3050466-af0e-4fd0-b670-ec6aac049086 admin admin] PATH_INFO: `/v3/projects/daa1d75d7376465eb9ea857de06b51ab/users/19eec26f1b9c4ab485d3bbc90dc7327a/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:43.025531 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3050466-af0e-4fd0-b670-ec6aac049086 admin admin] RBAC: Authorizing `identity:create_grant(project_id=daa1d75d7376465eb9ea857de06b51ab, user_id=19eec26f1b9c4ab485d3bbc90dc7327a, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:43.028796 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3050466-af0e-4fd0-b670-ec6aac049086 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:43.124476 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 960/1918] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:46:42 2022] PUT /identity/v3/projects/daa1d75d7376465eb9ea857de06b51ab/users/19eec26f1b9c4ab485d3bbc90dc7327a/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:43.125273 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f8c0c4-9081-49d2-8729-1e415282eba2 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:43.147658 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f8c0c4-9081-49d2-8729-1e415282eba2 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:43.208554 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:43.245594 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa39cd0-ae2e-473b-8305-252579552af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:43.247338 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa39cd0-ae2e-473b-8305-252579552af4 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:43.247338 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa39cd0-ae2e-473b-8305-252579552af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:43.247338 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa39cd0-ae2e-473b-8305-252579552af4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:43.249668 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa39cd0-ae2e-473b-8305-252579552af4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:43.251727 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa39cd0-ae2e-473b-8305-252579552af4 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:43.266125 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 959/1919] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:46:43 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:46:43.270717 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3050466-af0e-4fd0-b670-ec6aac049086 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:43.289539 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3050466-af0e-4fd0-b670-ec6aac049086 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:43.292598 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff70713-2b36-4ba7-87ca-cad0ddef749e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:43.293759 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff70713-2b36-4ba7-87ca-cad0ddef749e admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:43.294104 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff70713-2b36-4ba7-87ca-cad0ddef749e admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:43.294464 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff70713-2b36-4ba7-87ca-cad0ddef749e admin admin] PATH_INFO: `/v3/projects/daa1d75d7376465eb9ea857de06b51ab/users/19eec26f1b9c4ab485d3bbc90dc7327a/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:43.296822 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff70713-2b36-4ba7-87ca-cad0ddef749e admin admin] RBAC: Authorizing `identity:create_grant(project_id=daa1d75d7376465eb9ea857de06b51ab, user_id=19eec26f1b9c4ab485d3bbc90dc7327a, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:43.299660 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff70713-2b36-4ba7-87ca-cad0ddef749e admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:43.324351 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 961/1920] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:46:43 2022] PUT /identity/v3/projects/daa1d75d7376465eb9ea857de06b51ab/users/19eec26f1b9c4ab485d3bbc90dc7327a/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:43.327994 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ef91b-0f45-41df-8adf-f9fbf2eee6b0 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:43.328367 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ef91b-0f45-41df-8adf-f9fbf2eee6b0 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:43.328756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ef91b-0f45-41df-8adf-f9fbf2eee6b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:43.367246 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-5d6ef91b-0f45-41df-8adf-f9fbf2eee6b0 None None] MFA Rules not processed for user `19eec26f1b9c4ab485d3bbc90dc7327a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:43.368712 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-5d6ef91b-0f45-41df-8adf-f9fbf2eee6b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:43.485580 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 960/1921] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:43 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:43.503676 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff70713-2b36-4ba7-87ca-cad0ddef749e admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:43.509438 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-9ff70713-2b36-4ba7-87ca-cad0ddef749e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:43.583929 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff70713-2b36-4ba7-87ca-cad0ddef749e admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:43.590200 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968ba298-b50b-4eac-9edf-e90d91c1c50f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:43.590912 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ba298-b50b-4eac-9edf-e90d91c1c50f service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:43.591191 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ba298-b50b-4eac-9edf-e90d91c1c50f service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:43.591464 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ba298-b50b-4eac-9edf-e90d91c1c50f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:43.601180 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968ba298-b50b-4eac-9edf-e90d91c1c50f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:43.603277 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968ba298-b50b-4eac-9edf-e90d91c1c50f service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:43.621315 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 962/1922] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:46:43 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:46:44.282127 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:44.486355 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:44.716684 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6ef91b-0f45-41df-8adf-f9fbf2eee6b0 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:44.718705 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-5d6ef91b-0f45-41df-8adf-f9fbf2eee6b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:44.847347 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6ef91b-0f45-41df-8adf-f9fbf2eee6b0 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:44.852869 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981d8723-080f-4359-bddd-f3bd8d8f8dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:44.853923 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d8723-080f-4359-bddd-f3bd8d8f8dc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:44.854278 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d8723-080f-4359-bddd-f3bd8d8f8dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:44.854629 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d8723-080f-4359-bddd-f3bd8d8f8dc7 admin admin] PATH_INFO: `/v3/users/7f4e4df43d9d48f481d8be85380f5a58` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:44.856077 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981d8723-080f-4359-bddd-f3bd8d8f8dc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f4e4df43d9d48f481d8be85380f5a58)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:44.858120 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981d8723-080f-4359-bddd-f3bd8d8f8dc7 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:44.941234 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968ba298-b50b-4eac-9edf-e90d91c1c50f service cinder] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:44.942689 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-968ba298-b50b-4eac-9edf-e90d91c1c50f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:44.976501 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968ba298-b50b-4eac-9edf-e90d91c1c50f service cinder] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:44.982940 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70425344-e554-4705-9e1a-bd2214d0708e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:44.984998 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70425344-e554-4705-9e1a-bd2214d0708e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:44.984998 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70425344-e554-4705-9e1a-bd2214d0708e admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:44.984998 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70425344-e554-4705-9e1a-bd2214d0708e admin admin] PATH_INFO: `/v3/users/fd17b80bcb40420c9ed221d676bb5222` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:44.985887 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70425344-e554-4705-9e1a-bd2214d0708e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd17b80bcb40420c9ed221d676bb5222)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:44.987901 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70425344-e554-4705-9e1a-bd2214d0708e admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:45.038847 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-981d8723-080f-4359-bddd-f3bd8d8f8dc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f4e4df43d9d48f481d8be85380f5a58'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:45.122293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-981d8723-080f-4359-bddd-f3bd8d8f8dc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f4e4df43d9d48f481d8be85380f5a58'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:45.131242 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-981d8723-080f-4359-bddd-f3bd8d8f8dc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f4e4df43d9d48f481d8be85380f5a58'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:45.149104 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-981d8723-080f-4359-bddd-f3bd8d8f8dc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f4e4df43d9d48f481d8be85380f5a58'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:45.150311 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-981d8723-080f-4359-bddd-f3bd8d8f8dc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f4e4df43d9d48f481d8be85380f5a58'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:45.157359 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 961/1923] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:46:44 2022] DELETE /identity/v3/users/7f4e4df43d9d48f481d8be85380f5a58 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:45.158487 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981d8723-080f-4359-bddd-f3bd8d8f8dc7 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:45.159957 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-981d8723-080f-4359-bddd-f3bd8d8f8dc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:45.193023 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-70425344-e554-4705-9e1a-bd2214d0708e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd17b80bcb40420c9ed221d676bb5222'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:45.227631 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981d8723-080f-4359-bddd-f3bd8d8f8dc7 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:45.231847 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6365bd45-656c-4cbd-bfd9-f1f7d3f2da0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:45.232551 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6365bd45-656c-4cbd-bfd9-f1f7d3f2da0d service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:45.232842 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6365bd45-656c-4cbd-bfd9-f1f7d3f2da0d service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:45.233126 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6365bd45-656c-4cbd-bfd9-f1f7d3f2da0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:45.234925 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-6365bd45-656c-4cbd-bfd9-f1f7d3f2da0d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:45.309693 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6365bd45-656c-4cbd-bfd9-f1f7d3f2da0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:45.311677 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6365bd45-656c-4cbd-bfd9-f1f7d3f2da0d service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:45.323217 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 962/1924] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:46:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:46:45.331337 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc3f53-9518-411b-a684-ab954ca19f58 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:45.331680 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc3f53-9518-411b-a684-ab954ca19f58 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:45.331946 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc3f53-9518-411b-a684-ab954ca19f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:45.351392 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-dfdc3f53-9518-411b-a684-ab954ca19f58 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:45.353110 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-dfdc3f53-9518-411b-a684-ab954ca19f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:45.354270 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-70425344-e554-4705-9e1a-bd2214d0708e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd17b80bcb40420c9ed221d676bb5222'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:45.355338 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-70425344-e554-4705-9e1a-bd2214d0708e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd17b80bcb40420c9ed221d676bb5222'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:45.356453 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-70425344-e554-4705-9e1a-bd2214d0708e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd17b80bcb40420c9ed221d676bb5222'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:45.365173 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-70425344-e554-4705-9e1a-bd2214d0708e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd17b80bcb40420c9ed221d676bb5222'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:45.424258 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 963/1925] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:46:44 2022] DELETE /identity/v3/users/fd17b80bcb40420c9ed221d676bb5222 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:45.426031 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b68bd6-e29d-4f14-9a3f-444fff893672 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:45.437417 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b68bd6-e29d-4f14-9a3f-444fff893672 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:45.437417 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b68bd6-e29d-4f14-9a3f-444fff893672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:45.444309 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 963/1926] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:45.456130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc3f53-9518-411b-a684-ab954ca19f58 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:45.461900 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-dfdc3f53-9518-411b-a684-ab954ca19f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:45.462770 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-81b68bd6-e29d-4f14-9a3f-444fff893672 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:45.466185 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-81b68bd6-e29d-4f14-9a3f-444fff893672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:45.599421 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 964/1927] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:46:45.614773 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b68bd6-e29d-4f14-9a3f-444fff893672 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:45.630022 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc3f53-9518-411b-a684-ab954ca19f58 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:45.746548 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b68bd6-e29d-4f14-9a3f-444fff893672 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:45.750646 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88dee1d-d327-4d1a-8749-708f7e016f0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:45.751365 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88dee1d-d327-4d1a-8749-708f7e016f0e service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:45.752577 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88dee1d-d327-4d1a-8749-708f7e016f0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:45.752751 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96db5a79-c983-4880-b9ec-73a4cad01e2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:45.752921 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88dee1d-d327-4d1a-8749-708f7e016f0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:45.753077 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db5a79-c983-4880-b9ec-73a4cad01e2e service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:45.753326 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db5a79-c983-4880-b9ec-73a4cad01e2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:45.753907 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db5a79-c983-4880-b9ec-73a4cad01e2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:45.763826 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88dee1d-d327-4d1a-8749-708f7e016f0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:45.766616 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96db5a79-c983-4880-b9ec-73a4cad01e2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:45.767311 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88dee1d-d327-4d1a-8749-708f7e016f0e service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:45.775642 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96db5a79-c983-4880-b9ec-73a4cad01e2e service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:45.780143 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 965/1928] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:46:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:46:45.789414 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 964/1929] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:46:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:46:45.874627 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:46.259538 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:46.888241 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:47.154086 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88dee1d-d327-4d1a-8749-708f7e016f0e service neutron] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:47.156150 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-e88dee1d-d327-4d1a-8749-708f7e016f0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:47.165914 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:47.174019 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96db5a79-c983-4880-b9ec-73a4cad01e2e service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:47.175148 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-96db5a79-c983-4880-b9ec-73a4cad01e2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:47.186922 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88dee1d-d327-4d1a-8749-708f7e016f0e service neutron] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:47.194906 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85569b87-2dd1-4531-8739-b3ce2c744292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:47.195969 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85569b87-2dd1-4531-8739-b3ce2c744292 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:47.197448 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85569b87-2dd1-4531-8739-b3ce2c744292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:47.197932 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85569b87-2dd1-4531-8739-b3ce2c744292 admin admin] PATH_INFO: `/v3/projects/fbf459f524444ce8b3ebc7ca37f80468` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:47.199468 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85569b87-2dd1-4531-8739-b3ce2c744292 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbf459f524444ce8b3ebc7ca37f80468)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:47.202683 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85569b87-2dd1-4531-8739-b3ce2c744292 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:47.280419 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96db5a79-c983-4880-b9ec-73a4cad01e2e service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:47.303242 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26289a9-3ff1-45fb-9c65-d593ecc3c601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:47.304773 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26289a9-3ff1-45fb-9c65-d593ecc3c601 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:47.304773 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26289a9-3ff1-45fb-9c65-d593ecc3c601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:47.306645 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26289a9-3ff1-45fb-9c65-d593ecc3c601 admin admin] PATH_INFO: `/v3/projects/97eff8f126e34cb78c3f8fb66c5f3222` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:47.308692 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26289a9-3ff1-45fb-9c65-d593ecc3c601 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97eff8f126e34cb78c3f8fb66c5f3222)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:47.311401 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26289a9-3ff1-45fb-9c65-d593ecc3c601 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:47.402327 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-85569b87-2dd1-4531-8739-b3ce2c744292 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbf459f524444ce8b3ebc7ca37f80468', 'request_id': 'req-85569b87-2dd1-4531-8739-b3ce2c744292'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:47.419328 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-85569b87-2dd1-4531-8739-b3ce2c744292 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbf459f524444ce8b3ebc7ca37f80468', 'request_id': 'req-85569b87-2dd1-4531-8739-b3ce2c744292'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:47.474054 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:47.506339 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b26289a9-3ff1-45fb-9c65-d593ecc3c601 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97eff8f126e34cb78c3f8fb66c5f3222', 'request_id': 'req-b26289a9-3ff1-45fb-9c65-d593ecc3c601'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:47.518282 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b26289a9-3ff1-45fb-9c65-d593ecc3c601 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97eff8f126e34cb78c3f8fb66c5f3222', 'request_id': 'req-b26289a9-3ff1-45fb-9c65-d593ecc3c601'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:47.522236 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-85569b87-2dd1-4531-8739-b3ce2c744292 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbf459f524444ce8b3ebc7ca37f80468', 'request_id': 'req-85569b87-2dd1-4531-8739-b3ce2c744292'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:47.548762 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b26289a9-3ff1-45fb-9c65-d593ecc3c601 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97eff8f126e34cb78c3f8fb66c5f3222', 'request_id': 'req-b26289a9-3ff1-45fb-9c65-d593ecc3c601'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:47.572256 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-85569b87-2dd1-4531-8739-b3ce2c744292 admin admin] The token cache is being invalidate because project fbf459f524444ce8b3ebc7ca37f80468 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:46:47.577437 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-85569b87-2dd1-4531-8739-b3ce2c744292 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:47.577781 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-85569b87-2dd1-4531-8739-b3ce2c744292 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:47.581389 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 966/1930] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:46:47 2022] DELETE /identity/v3/projects/fbf459f524444ce8b3ebc7ca37f80468 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:47.594355 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22522e3e-e0ee-4008-9b92-b6ca5a40a843 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:47.594946 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22522e3e-e0ee-4008-9b92-b6ca5a40a843 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:47.595450 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22522e3e-e0ee-4008-9b92-b6ca5a40a843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:47.626846 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b26289a9-3ff1-45fb-9c65-d593ecc3c601 admin admin] The token cache is being invalidate because project 97eff8f126e34cb78c3f8fb66c5f3222 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:46:47.627218 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b26289a9-3ff1-45fb-9c65-d593ecc3c601 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:47.628128 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b26289a9-3ff1-45fb-9c65-d593ecc3c601 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:47.629233 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-22522e3e-e0ee-4008-9b92-b6ca5a40a843 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:47.631374 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 965/1931] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:46:47 2022] DELETE /identity/v3/projects/97eff8f126e34cb78c3f8fb66c5f3222 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:46:47.636717 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-22522e3e-e0ee-4008-9b92-b6ca5a40a843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:47.636960 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26289a9-3ff1-45fb-9c65-d593ecc3c601 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:47.640499 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-b26289a9-3ff1-45fb-9c65-d593ecc3c601 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:47.744878 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 967/1932] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:47.746488 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a545ffc-094b-4de1-a9fc-6170523eced9 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:47.747451 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a545ffc-094b-4de1-a9fc-6170523eced9 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:47.747893 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a545ffc-094b-4de1-a9fc-6170523eced9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:47.775558 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-0a545ffc-094b-4de1-a9fc-6170523eced9 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:47.779583 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-0a545ffc-094b-4de1-a9fc-6170523eced9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:47.800116 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26289a9-3ff1-45fb-9c65-d593ecc3c601 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:47.804107 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 968/1933] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:47.806425 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ff95c7-52da-4ebe-a328-7f73a0d23c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:47.809037 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a545ffc-094b-4de1-a9fc-6170523eced9 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:47.811693 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff95c7-52da-4ebe-a328-7f73a0d23c93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:47.812263 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff95c7-52da-4ebe-a328-7f73a0d23c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:47.812753 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff95c7-52da-4ebe-a328-7f73a0d23c93 admin admin] PATH_INFO: `/v3/users/1d56e2c90cfd4cfca5714dfefab9bdd3` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:47.814460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ff95c7-52da-4ebe-a328-7f73a0d23c93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d56e2c90cfd4cfca5714dfefab9bdd3)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:47.817053 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ff95c7-52da-4ebe-a328-7f73a0d23c93 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:47.826280 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a545ffc-094b-4de1-a9fc-6170523eced9 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:47.833849 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaa224b-650a-4542-ba1b-3d0004d56554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:47.834545 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa224b-650a-4542-ba1b-3d0004d56554 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:47.835370 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa224b-650a-4542-ba1b-3d0004d56554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:47.835370 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa224b-650a-4542-ba1b-3d0004d56554 admin admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:47.836037 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaa224b-650a-4542-ba1b-3d0004d56554 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:46:47.836493 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaa224b-650a-4542-ba1b-3d0004d56554 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:47.838636 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaa224b-650a-4542-ba1b-3d0004d56554 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:47.863590 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 969/1934] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 16:46:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:46:47.867782 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaa224b-650a-4542-ba1b-3d0004d56554 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:47.881603 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaa224b-650a-4542-ba1b-3d0004d56554 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:47.897410 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c0f148-83d8-4016-bd22-b4f5fb92b745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:47.899375 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c0f148-83d8-4016-bd22-b4f5fb92b745 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:47.900759 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c0f148-83d8-4016-bd22-b4f5fb92b745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:47.901121 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c0f148-83d8-4016-bd22-b4f5fb92b745 admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:47.903248 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c0f148-83d8-4016-bd22-b4f5fb92b745 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:47.905331 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c0f148-83d8-4016-bd22-b4f5fb92b745 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:48.025996 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 970/1935] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:46:47 2022] POST /identity/v3/projects => generated 373 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:46:48.026854 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-62ff95c7-52da-4ebe-a328-7f73a0d23c93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d56e2c90cfd4cfca5714dfefab9bdd3'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:48.028382 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c0f148-83d8-4016-bd22-b4f5fb92b745 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:48.087654 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-62ff95c7-52da-4ebe-a328-7f73a0d23c93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d56e2c90cfd4cfca5714dfefab9bdd3'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:48.094553 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-62ff95c7-52da-4ebe-a328-7f73a0d23c93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d56e2c90cfd4cfca5714dfefab9bdd3'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:48.128613 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-62ff95c7-52da-4ebe-a328-7f73a0d23c93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d56e2c90cfd4cfca5714dfefab9bdd3'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:48.129477 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-62ff95c7-52da-4ebe-a328-7f73a0d23c93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d56e2c90cfd4cfca5714dfefab9bdd3'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:48.132365 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c0f148-83d8-4016-bd22-b4f5fb92b745 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:48.140982 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c2cd54-399d-41c4-aaf3-7aa7726b72c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:48.140982 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c2cd54-399d-41c4-aaf3-7aa7726b72c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:48.140982 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c2cd54-399d-41c4-aaf3-7aa7726b72c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:48.142643 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 966/1936] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:46:47 2022] DELETE /identity/v3/users/1d56e2c90cfd4cfca5714dfefab9bdd3 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:46:48.144472 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c2cd54-399d-41c4-aaf3-7aa7726b72c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:48.145670 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c2cd54-399d-41c4-aaf3-7aa7726b72c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:48.147786 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c2cd54-399d-41c4-aaf3-7aa7726b72c0 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:48.240344 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 971/1937] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:46:48 2022] POST /identity/v3/users => generated 287 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:46:48.245560 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ff95c7-52da-4ebe-a328-7f73a0d23c93 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:48.247737 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-62ff95c7-52da-4ebe-a328-7f73a0d23c93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:48.349318 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ff95c7-52da-4ebe-a328-7f73a0d23c93 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:48.357070 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ff28c1-30a8-4ace-8bc1-8a31e2e8ae59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:48.361131 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff28c1-30a8-4ace-8bc1-8a31e2e8ae59 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:48.361131 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff28c1-30a8-4ace-8bc1-8a31e2e8ae59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:48.361131 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff28c1-30a8-4ace-8bc1-8a31e2e8ae59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:48.361131 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ff28c1-30a8-4ace-8bc1-8a31e2e8ae59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:48.361131 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ff28c1-30a8-4ace-8bc1-8a31e2e8ae59 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:48.379225 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 967/1938] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:46:48 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:46:48.383775 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c2cd54-399d-41c4-aaf3-7aa7726b72c0 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:48.398263 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c2cd54-399d-41c4-aaf3-7aa7726b72c0 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:48.405746 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823fc070-8829-4708-aba8-1cb111f54618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:48.407196 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823fc070-8829-4708-aba8-1cb111f54618 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:48.407883 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823fc070-8829-4708-aba8-1cb111f54618 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:48.408172 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823fc070-8829-4708-aba8-1cb111f54618 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:48.409432 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823fc070-8829-4708-aba8-1cb111f54618 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:48.411580 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823fc070-8829-4708-aba8-1cb111f54618 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:48.428754 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 972/1939] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:46:48 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:46:48.434796 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ff28c1-30a8-4ace-8bc1-8a31e2e8ae59 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:48.461535 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ff28c1-30a8-4ace-8bc1-8a31e2e8ae59 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:48.467378 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95454d3d-56ee-4f0f-9a8c-95794fb5fe12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:48.468705 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95454d3d-56ee-4f0f-9a8c-95794fb5fe12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:48.469230 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95454d3d-56ee-4f0f-9a8c-95794fb5fe12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:48.469725 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95454d3d-56ee-4f0f-9a8c-95794fb5fe12 admin admin] PATH_INFO: `/v3/projects/5d70fe1437fc423681962ee747f07410/users/6472100830a54d6a8e21ee55b68e6a02/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:48.493447 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95454d3d-56ee-4f0f-9a8c-95794fb5fe12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d70fe1437fc423681962ee747f07410, user_id=6472100830a54d6a8e21ee55b68e6a02, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:48.497569 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95454d3d-56ee-4f0f-9a8c-95794fb5fe12 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:48.605661 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 968/1940] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:46:48 2022] PUT /identity/v3/projects/5d70fe1437fc423681962ee747f07410/users/6472100830a54d6a8e21ee55b68e6a02/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:48.612379 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701387be-2916-4eed-bc58-edad7a20c375 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:48.612996 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701387be-2916-4eed-bc58-edad7a20c375 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:48.613805 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701387be-2916-4eed-bc58-edad7a20c375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:48.653888 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-701387be-2916-4eed-bc58-edad7a20c375 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:48.656807 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-701387be-2916-4eed-bc58-edad7a20c375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:48.749072 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 973/1941] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:48.771177 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95454d3d-56ee-4f0f-9a8c-95794fb5fe12 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:48.773370 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-95454d3d-56ee-4f0f-9a8c-95794fb5fe12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:48.867890 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95454d3d-56ee-4f0f-9a8c-95794fb5fe12 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:48.873308 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f7cac4-81e6-4ffb-82aa-367f43f07118 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:48.874025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7cac4-81e6-4ffb-82aa-367f43f07118 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:48.874526 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7cac4-81e6-4ffb-82aa-367f43f07118 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:48.875030 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7cac4-81e6-4ffb-82aa-367f43f07118 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:48.890189 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f7cac4-81e6-4ffb-82aa-367f43f07118 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:48.892485 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f7cac4-81e6-4ffb-82aa-367f43f07118 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:48.907468 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 969/1942] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:46:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:46:48.921631 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701387be-2916-4eed-bc58-edad7a20c375 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:48.929727 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-701387be-2916-4eed-bc58-edad7a20c375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:48.955832 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701387be-2916-4eed-bc58-edad7a20c375 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:48.968261 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fec369-4f36-49a4-b826-36a3d1db155d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:48.970197 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fec369-4f36-49a4-b826-36a3d1db155d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:48.970584 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fec369-4f36-49a4-b826-36a3d1db155d admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:48.971010 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fec369-4f36-49a4-b826-36a3d1db155d admin admin] PATH_INFO: `/v3/projects/5d00af09b6334ef2b8f7b06fb27c3063` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:48.980834 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fec369-4f36-49a4-b826-36a3d1db155d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d00af09b6334ef2b8f7b06fb27c3063)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:48.983140 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fec369-4f36-49a4-b826-36a3d1db155d admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:49.089638 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:49.187907 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-e2fec369-4f36-49a4-b826-36a3d1db155d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d00af09b6334ef2b8f7b06fb27c3063', 'request_id': 'req-e2fec369-4f36-49a4-b826-36a3d1db155d'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:49.198918 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-e2fec369-4f36-49a4-b826-36a3d1db155d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d00af09b6334ef2b8f7b06fb27c3063', 'request_id': 'req-e2fec369-4f36-49a4-b826-36a3d1db155d'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:49.282387 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-e2fec369-4f36-49a4-b826-36a3d1db155d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d00af09b6334ef2b8f7b06fb27c3063', 'request_id': 'req-e2fec369-4f36-49a4-b826-36a3d1db155d'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:49.303196 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-e2fec369-4f36-49a4-b826-36a3d1db155d admin admin] The token cache is being invalidate because project 5d00af09b6334ef2b8f7b06fb27c3063 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:46:49.303596 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-e2fec369-4f36-49a4-b826-36a3d1db155d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:49.304416 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-e2fec369-4f36-49a4-b826-36a3d1db155d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:49.308028 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 974/1943] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:46:48 2022] DELETE /identity/v3/projects/5d00af09b6334ef2b8f7b06fb27c3063 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:49.313747 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf5cf2-10cb-4c3b-921c-d548916eeb5c None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:49.314755 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf5cf2-10cb-4c3b-921c-d548916eeb5c None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:49.315256 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf5cf2-10cb-4c3b-921c-d548916eeb5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:49.338877 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-c5bf5cf2-10cb-4c3b-921c-d548916eeb5c None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:49.340196 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-c5bf5cf2-10cb-4c3b-921c-d548916eeb5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:49.410049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 970/1944] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:49.410836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:49.420074 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe0d7c-ad6e-493b-bdba-0a904c405e1f None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:49.420612 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe0d7c-ad6e-493b-bdba-0a904c405e1f None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:49.421036 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe0d7c-ad6e-493b-bdba-0a904c405e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:49.442878 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-5dfe0d7c-ad6e-493b-bdba-0a904c405e1f None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:49.444405 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-5dfe0d7c-ad6e-493b-bdba-0a904c405e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:49.468192 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 975/1945] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:46:49.468523 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bf5cf2-10cb-4c3b-921c-d548916eeb5c None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:49.472764 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bf5cf2-10cb-4c3b-921c-d548916eeb5c None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:49.478275 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fc3182-1c39-4cb5-9a00-b68e732f0de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:49.478947 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc3182-1c39-4cb5-9a00-b68e732f0de8 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:49.479379 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc3182-1c39-4cb5-9a00-b68e732f0de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:49.479935 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc3182-1c39-4cb5-9a00-b68e732f0de8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:49.480911 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fc3182-1c39-4cb5-9a00-b68e732f0de8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:46:49.481618 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fc3182-1c39-4cb5-9a00-b68e732f0de8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:49.485236 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fc3182-1c39-4cb5-9a00-b68e732f0de8 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:49.507428 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 971/1946] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 16:46:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:46:49.512698 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe0d7c-ad6e-493b-bdba-0a904c405e1f None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:49.522890 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe0d7c-ad6e-493b-bdba-0a904c405e1f None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:49.531932 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab83df3-cfb9-43a1-96e6-59fd90d5c424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:49.532958 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab83df3-cfb9-43a1-96e6-59fd90d5c424 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:49.533240 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab83df3-cfb9-43a1-96e6-59fd90d5c424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:49.533517 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab83df3-cfb9-43a1-96e6-59fd90d5c424 admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:49.534218 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab83df3-cfb9-43a1-96e6-59fd90d5c424 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:49.536601 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab83df3-cfb9-43a1-96e6-59fd90d5c424 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:49.643985 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 976/1947] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:46:49 2022] POST /identity/v3/projects => generated 393 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:46:49.645758 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fc3182-1c39-4cb5-9a00-b68e732f0de8 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:49.662149 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fc3182-1c39-4cb5-9a00-b68e732f0de8 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:49.778274 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4037c7-465a-40f7-b66c-8ac8ff3398ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:49.779014 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4037c7-465a-40f7-b66c-8ac8ff3398ee admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:49.779306 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4037c7-465a-40f7-b66c-8ac8ff3398ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:49.779739 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4037c7-465a-40f7-b66c-8ac8ff3398ee admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:49.780473 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4037c7-465a-40f7-b66c-8ac8ff3398ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:49.786628 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4037c7-465a-40f7-b66c-8ac8ff3398ee admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:49.905331 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 972/1948] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:46:49 2022] POST /identity/v3/users => generated 297 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:46:49.909336 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab83df3-cfb9-43a1-96e6-59fd90d5c424 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:49.932162 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab83df3-cfb9-43a1-96e6-59fd90d5c424 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:49.939892 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7372ef31-8469-4122-a417-21b6ce21044d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:49.940613 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7372ef31-8469-4122-a417-21b6ce21044d admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:49.940895 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7372ef31-8469-4122-a417-21b6ce21044d admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:49.941181 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7372ef31-8469-4122-a417-21b6ce21044d admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:49.941933 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7372ef31-8469-4122-a417-21b6ce21044d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:49.944636 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7372ef31-8469-4122-a417-21b6ce21044d admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:49.960871 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 977/1949] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:46:49 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:46:49.970129 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4037c7-465a-40f7-b66c-8ac8ff3398ee admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:49.981269 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4037c7-465a-40f7-b66c-8ac8ff3398ee admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:49.987650 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e82c60-f3e4-43f4-be0e-a37000ea6f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:49.988342 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e82c60-f3e4-43f4-be0e-a37000ea6f95 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:49.988706 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e82c60-f3e4-43f4-be0e-a37000ea6f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:49.988913 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e82c60-f3e4-43f4-be0e-a37000ea6f95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:49.989647 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e82c60-f3e4-43f4-be0e-a37000ea6f95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:49.991269 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e82c60-f3e4-43f4-be0e-a37000ea6f95 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:50.002786 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 973/1950] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:46:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:46:50.007298 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7372ef31-8469-4122-a417-21b6ce21044d admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:50.020098 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7372ef31-8469-4122-a417-21b6ce21044d admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:50.025214 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48990654-4651-49a4-aed1-380316389cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:50.027208 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48990654-4651-49a4-aed1-380316389cfb admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:50.027208 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48990654-4651-49a4-aed1-380316389cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:50.027208 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48990654-4651-49a4-aed1-380316389cfb admin admin] PATH_INFO: `/v3/projects/6017e4b8ab1e4f2fbdcf016ad3abce9f/users/b4df0c1223e04445a64cdf931aa48baf/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:50.047852 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48990654-4651-49a4-aed1-380316389cfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6017e4b8ab1e4f2fbdcf016ad3abce9f, user_id=b4df0c1223e04445a64cdf931aa48baf, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:50.048042 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48990654-4651-49a4-aed1-380316389cfb admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:50.130635 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 978/1951] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:46:50 2022] PUT /identity/v3/projects/6017e4b8ab1e4f2fbdcf016ad3abce9f/users/b4df0c1223e04445a64cdf931aa48baf/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:50.144074 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68174d1-1017-430e-9737-0900ba56cf83 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:50.144366 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68174d1-1017-430e-9737-0900ba56cf83 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:50.144652 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68174d1-1017-430e-9737-0900ba56cf83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:50.173946 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-e68174d1-1017-430e-9737-0900ba56cf83 None None] MFA Rules not processed for user `b4df0c1223e04445a64cdf931aa48baf`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:50.175177 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e68174d1-1017-430e-9737-0900ba56cf83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:50.296526 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 974/1952] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:50.303850 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48990654-4651-49a4-aed1-380316389cfb admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:50.385718 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48990654-4651-49a4-aed1-380316389cfb admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:50.390515 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab75b58-4372-4b84-90aa-c0d941fd2f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:50.391247 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab75b58-4372-4b84-90aa-c0d941fd2f99 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:50.391574 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab75b58-4372-4b84-90aa-c0d941fd2f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:50.391853 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab75b58-4372-4b84-90aa-c0d941fd2f99 admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:50.392557 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab75b58-4372-4b84-90aa-c0d941fd2f99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:50.395360 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab75b58-4372-4b84-90aa-c0d941fd2f99 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:50.521757 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 979/1953] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:46:50 2022] POST /identity/v3/projects => generated 393 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:46:50.527813 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68174d1-1017-430e-9737-0900ba56cf83 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:50.541104 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68174d1-1017-430e-9737-0900ba56cf83 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:50.628755 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d816d1e-ff20-4cef-9476-ecba22dec0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:50.629448 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d816d1e-ff20-4cef-9476-ecba22dec0ba admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:50.629732 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d816d1e-ff20-4cef-9476-ecba22dec0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:50.630069 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d816d1e-ff20-4cef-9476-ecba22dec0ba admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:50.630703 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d816d1e-ff20-4cef-9476-ecba22dec0ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:50.632884 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d816d1e-ff20-4cef-9476-ecba22dec0ba admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:50.763351 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 975/1954] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:46:50 2022] POST /identity/v3/users => generated 303 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:46:50.776455 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab75b58-4372-4b84-90aa-c0d941fd2f99 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:50.804695 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab75b58-4372-4b84-90aa-c0d941fd2f99 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:50.824829 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3070d3bb-1485-4172-b5d9-2b25c8b677b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:50.826111 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070d3bb-1485-4172-b5d9-2b25c8b677b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:50.826672 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070d3bb-1485-4172-b5d9-2b25c8b677b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:50.827207 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070d3bb-1485-4172-b5d9-2b25c8b677b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:50.828322 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3070d3bb-1485-4172-b5d9-2b25c8b677b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:50.831920 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3070d3bb-1485-4172-b5d9-2b25c8b677b6 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:50.865528 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 980/1955] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:46:50 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:46:50.866950 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d816d1e-ff20-4cef-9476-ecba22dec0ba admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:50.900552 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d816d1e-ff20-4cef-9476-ecba22dec0ba admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:50.932355 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd9669d-44cc-4cb1-86f7-fbcda5707e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:50.933853 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9669d-44cc-4cb1-86f7-fbcda5707e4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:50.934143 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9669d-44cc-4cb1-86f7-fbcda5707e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:50.934891 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9669d-44cc-4cb1-86f7-fbcda5707e4c admin admin] PATH_INFO: `/v3/projects/a08689affad24ad388a204ec66fc23e5/users/3c199f3170ec44faab7464a13071092e/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:50.967370 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd9669d-44cc-4cb1-86f7-fbcda5707e4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a08689affad24ad388a204ec66fc23e5, user_id=3c199f3170ec44faab7464a13071092e, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:50.970908 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd9669d-44cc-4cb1-86f7-fbcda5707e4c admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:51.087266 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 976/1956] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:46:50 2022] PUT /identity/v3/projects/a08689affad24ad388a204ec66fc23e5/users/3c199f3170ec44faab7464a13071092e/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:51.094814 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3070d3bb-1485-4172-b5d9-2b25c8b677b6 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:51.116159 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3070d3bb-1485-4172-b5d9-2b25c8b677b6 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:51.166808 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:51.243704 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd9669d-44cc-4cb1-86f7-fbcda5707e4c admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:51.245971 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-1cd9669d-44cc-4cb1-86f7-fbcda5707e4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:51.279806 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c657821-b19a-41a6-8678-2bb76609bdb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:51.281188 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c657821-b19a-41a6-8678-2bb76609bdb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:51.281516 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c657821-b19a-41a6-8678-2bb76609bdb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:51.281799 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c657821-b19a-41a6-8678-2bb76609bdb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:51.283203 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c657821-b19a-41a6-8678-2bb76609bdb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:51.285522 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c657821-b19a-41a6-8678-2bb76609bdb6 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:51.318523 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 981/1957] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:46:51 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:46:51.320660 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c657821-b19a-41a6-8678-2bb76609bdb6 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:51.339877 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c657821-b19a-41a6-8678-2bb76609bdb6 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:51.348548 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc77f65-78dc-45f6-b38e-1f51015c9e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:51.348548 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc77f65-78dc-45f6-b38e-1f51015c9e33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:51.348969 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc77f65-78dc-45f6-b38e-1f51015c9e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:51.349466 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc77f65-78dc-45f6-b38e-1f51015c9e33 admin admin] PATH_INFO: `/v3/projects/a08689affad24ad388a204ec66fc23e5/users/3c199f3170ec44faab7464a13071092e/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:51.353629 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc77f65-78dc-45f6-b38e-1f51015c9e33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a08689affad24ad388a204ec66fc23e5, user_id=3c199f3170ec44faab7464a13071092e, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:51.357507 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc77f65-78dc-45f6-b38e-1f51015c9e33 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:51.379225 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 982/1958] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:46:51 2022] PUT /identity/v3/projects/a08689affad24ad388a204ec66fc23e5/users/3c199f3170ec44faab7464a13071092e/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:51.390641 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd9669d-44cc-4cb1-86f7-fbcda5707e4c admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:51.393207 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd156ed6-a592-4989-ab2b-35d8b3ea872a None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:51.393496 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd156ed6-a592-4989-ab2b-35d8b3ea872a None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:51.393775 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd156ed6-a592-4989-ab2b-35d8b3ea872a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:51.459368 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-fd156ed6-a592-4989-ab2b-35d8b3ea872a None None] MFA Rules not processed for user `3c199f3170ec44faab7464a13071092e`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:51.461933 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-fd156ed6-a592-4989-ab2b-35d8b3ea872a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:51.479729 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb727b-810f-41f6-a3af-b15cc7d71a5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:51.480718 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb727b-810f-41f6-a3af-b15cc7d71a5d service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:51.481071 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb727b-810f-41f6-a3af-b15cc7d71a5d service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:51.481415 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb727b-810f-41f6-a3af-b15cc7d71a5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:51.483651 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e0fb727b-810f-41f6-a3af-b15cc7d71a5d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:51.565076 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fb727b-810f-41f6-a3af-b15cc7d71a5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:51.566918 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fb727b-810f-41f6-a3af-b15cc7d71a5d service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:51.584131 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 977/1959] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:46:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:46:51.622387 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 983/1960] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:51 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:51.634924 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb727b-810f-41f6-a3af-b15cc7d71a5d service cinder] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:51.637141 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e0fb727b-810f-41f6-a3af-b15cc7d71a5d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:51.652889 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:51.657163 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb727b-810f-41f6-a3af-b15cc7d71a5d service cinder] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:51.671798 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3185fb25-42c2-4a85-8982-cdab64a79ffa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:51.672483 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3185fb25-42c2-4a85-8982-cdab64a79ffa service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:51.674972 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3185fb25-42c2-4a85-8982-cdab64a79ffa service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:51.675482 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3185fb25-42c2-4a85-8982-cdab64a79ffa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:51.712323 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3185fb25-42c2-4a85-8982-cdab64a79ffa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:51.713946 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3185fb25-42c2-4a85-8982-cdab64a79ffa service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:51.738939 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 978/1961] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:46:51 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:46:51.856822 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd156ed6-a592-4989-ab2b-35d8b3ea872a None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:51.867549 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd156ed6-a592-4989-ab2b-35d8b3ea872a None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:51.872851 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dd2bbd-d979-4084-b42a-67f5da95984f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:51.873942 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd2bbd-d979-4084-b42a-67f5da95984f service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:51.874339 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd2bbd-d979-4084-b42a-67f5da95984f service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:51.874622 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd2bbd-d979-4084-b42a-67f5da95984f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:52.021202 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dd2bbd-d979-4084-b42a-67f5da95984f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:52.023364 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dd2bbd-d979-4084-b42a-67f5da95984f service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:52.036060 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 984/1962] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:46:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:46:53.080906 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:53.569896 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:54.781883 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:56.396745 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:56.526841 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4864c6db-8301-435b-8a1b-1060a65337bc None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:56.527057 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4864c6db-8301-435b-8a1b-1060a65337bc None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:56.527347 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4864c6db-8301-435b-8a1b-1060a65337bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:56.562257 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-4864c6db-8301-435b-8a1b-1060a65337bc None None] MFA Rules not processed for user `6472100830a54d6a8e21ee55b68e6a02`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:56.563528 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-4864c6db-8301-435b-8a1b-1060a65337bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:56.657565 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 979/1963] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:56.662227 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dd2bbd-d979-4084-b42a-67f5da95984f service cinder] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:56.663681 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-e6dd2bbd-d979-4084-b42a-67f5da95984f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:56.746564 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dd2bbd-d979-4084-b42a-67f5da95984f service cinder] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:56.752495 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdb5005-98b6-4b22-ae04-fcb94ad91cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:56.753247 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdb5005-98b6-4b22-ae04-fcb94ad91cc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:56.753595 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdb5005-98b6-4b22-ae04-fcb94ad91cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:56.753878 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdb5005-98b6-4b22-ae04-fcb94ad91cc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:56.754578 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdb5005-98b6-4b22-ae04-fcb94ad91cc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:56.756625 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdb5005-98b6-4b22-ae04-fcb94ad91cc8 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:56.877951 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 985/1964] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:46:56 2022] POST /identity/v3/projects => generated 361 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:46:56.884107 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4864c6db-8301-435b-8a1b-1060a65337bc None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:56.902664 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4864c6db-8301-435b-8a1b-1060a65337bc None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:56.982810 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a3a63b-5930-40c7-bb82-336435dc7fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:56.983563 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a3a63b-5930-40c7-bb82-336435dc7fd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:56.983857 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a3a63b-5930-40c7-bb82-336435dc7fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:56.984130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a3a63b-5930-40c7-bb82-336435dc7fd3 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:56.984855 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a3a63b-5930-40c7-bb82-336435dc7fd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:56.991919 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a3a63b-5930-40c7-bb82-336435dc7fd3 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:57.093777 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 980/1965] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:46:56 2022] POST /identity/v3/users => generated 287 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:46:57.095153 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdb5005-98b6-4b22-ae04-fcb94ad91cc8 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:57.106983 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdb5005-98b6-4b22-ae04-fcb94ad91cc8 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:57.112297 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22227ce4-8b46-4e6f-83cb-a5c16de6cf4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:57.112980 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22227ce4-8b46-4e6f-83cb-a5c16de6cf4a admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:57.113258 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22227ce4-8b46-4e6f-83cb-a5c16de6cf4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:57.113530 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22227ce4-8b46-4e6f-83cb-a5c16de6cf4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:57.114263 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22227ce4-8b46-4e6f-83cb-a5c16de6cf4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:57.116005 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22227ce4-8b46-4e6f-83cb-a5c16de6cf4a admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:57.128541 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 986/1966] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:46:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:46:57.133066 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a3a63b-5930-40c7-bb82-336435dc7fd3 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:57.145781 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a3a63b-5930-40c7-bb82-336435dc7fd3 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:57.151396 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68578b7-ffcf-4b3c-8324-978b255f7825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:57.152334 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68578b7-ffcf-4b3c-8324-978b255f7825 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:57.152615 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68578b7-ffcf-4b3c-8324-978b255f7825 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:57.152890 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68578b7-ffcf-4b3c-8324-978b255f7825 admin admin] PATH_INFO: `/v3/projects/13e5cd3bf86848afaec07d84e3fc0f69/users/9e337289fc9d40e5877f6f3adc8fd8ad/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:57.172058 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68578b7-ffcf-4b3c-8324-978b255f7825 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13e5cd3bf86848afaec07d84e3fc0f69, user_id=9e337289fc9d40e5877f6f3adc8fd8ad, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:57.174920 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68578b7-ffcf-4b3c-8324-978b255f7825 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:57.269469 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 981/1967] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:46:57 2022] PUT /identity/v3/projects/13e5cd3bf86848afaec07d84e3fc0f69/users/9e337289fc9d40e5877f6f3adc8fd8ad/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:46:57.273162 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22227ce4-8b46-4e6f-83cb-a5c16de6cf4a admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:57.284223 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22227ce4-8b46-4e6f-83cb-a5c16de6cf4a admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:57.369870 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dab23e7-603a-407a-a37c-2ecd4a0c1128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:57.370628 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab23e7-603a-407a-a37c-2ecd4a0c1128 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:57.370920 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab23e7-603a-407a-a37c-2ecd4a0c1128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:57.371197 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab23e7-603a-407a-a37c-2ecd4a0c1128 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:57.371998 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dab23e7-603a-407a-a37c-2ecd4a0c1128 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:57.373781 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dab23e7-603a-407a-a37c-2ecd4a0c1128 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:57.385326 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 987/1968] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:46:57 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:46:57.389237 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68578b7-ffcf-4b3c-8324-978b255f7825 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:57.401514 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68578b7-ffcf-4b3c-8324-978b255f7825 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:57.408081 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3f9882-f44f-46a3-bac6-7290a5e67f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:57.409163 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f9882-f44f-46a3-bac6-7290a5e67f08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:57.409433 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f9882-f44f-46a3-bac6-7290a5e67f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:57.409871 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f9882-f44f-46a3-bac6-7290a5e67f08 admin admin] PATH_INFO: `/v3/projects/13e5cd3bf86848afaec07d84e3fc0f69/users/9e337289fc9d40e5877f6f3adc8fd8ad/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:57.413203 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3f9882-f44f-46a3-bac6-7290a5e67f08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13e5cd3bf86848afaec07d84e3fc0f69, user_id=9e337289fc9d40e5877f6f3adc8fd8ad, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:57.416934 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3f9882-f44f-46a3-bac6-7290a5e67f08 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:57.440155 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 982/1969] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:46:57 2022] PUT /identity/v3/projects/13e5cd3bf86848afaec07d84e3fc0f69/users/9e337289fc9d40e5877f6f3adc8fd8ad/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:57.463003 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:57.486657 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dab23e7-603a-407a-a37c-2ecd4a0c1128 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:57.488573 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-4dab23e7-603a-407a-a37c-2ecd4a0c1128 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:57.502658 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dab23e7-603a-407a-a37c-2ecd4a0c1128 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:57.569761 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a9a345-0a4c-4dee-b0d5-1ac9e872a162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:57.570715 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a9a345-0a4c-4dee-b0d5-1ac9e872a162 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:57.571075 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a9a345-0a4c-4dee-b0d5-1ac9e872a162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:57.571357 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a9a345-0a4c-4dee-b0d5-1ac9e872a162 admin admin] PATH_INFO: `/v3/users/6ac6b0957d4c43589a1e9289a9c71117` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:57.573191 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a9a345-0a4c-4dee-b0d5-1ac9e872a162 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ac6b0957d4c43589a1e9289a9c71117)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:57.575172 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a9a345-0a4c-4dee-b0d5-1ac9e872a162 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:57.714419 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:57.808618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-a2a9a345-0a4c-4dee-b0d5-1ac9e872a162 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ac6b0957d4c43589a1e9289a9c71117'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:57.883146 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-a2a9a345-0a4c-4dee-b0d5-1ac9e872a162 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ac6b0957d4c43589a1e9289a9c71117'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:57.883869 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-a2a9a345-0a4c-4dee-b0d5-1ac9e872a162 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ac6b0957d4c43589a1e9289a9c71117'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:57.884688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-a2a9a345-0a4c-4dee-b0d5-1ac9e872a162 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ac6b0957d4c43589a1e9289a9c71117'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:57.900490 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-a2a9a345-0a4c-4dee-b0d5-1ac9e872a162 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ac6b0957d4c43589a1e9289a9c71117'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:57.925781 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 988/1970] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:46:57 2022] DELETE /identity/v3/users/6ac6b0957d4c43589a1e9289a9c71117 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:57.930244 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d135109-0f69-47dc-8739-2edadc964bae None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:57.932122 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d135109-0f69-47dc-8739-2edadc964bae None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:57.932731 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d135109-0f69-47dc-8739-2edadc964bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:57.956171 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-7d135109-0f69-47dc-8739-2edadc964bae None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:57.957407 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-7d135109-0f69-47dc-8739-2edadc964bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:58.038704 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 983/1971] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:46:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:58.051132 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a9a345-0a4c-4dee-b0d5-1ac9e872a162 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:58.052624 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-a2a9a345-0a4c-4dee-b0d5-1ac9e872a162 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:58.128947 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a9a345-0a4c-4dee-b0d5-1ac9e872a162 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:58.132683 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1760b6e-ab3d-4b00-ac5f-e25f6fc1923e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:58.133345 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1760b6e-ab3d-4b00-ac5f-e25f6fc1923e service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:58.134239 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1760b6e-ab3d-4b00-ac5f-e25f6fc1923e service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:58.135193 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1760b6e-ab3d-4b00-ac5f-e25f6fc1923e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:58.145624 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1760b6e-ab3d-4b00-ac5f-e25f6fc1923e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:58.147039 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1760b6e-ab3d-4b00-ac5f-e25f6fc1923e service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:58.158978 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 989/1972] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:46:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:46:58.902138 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d135109-0f69-47dc-8739-2edadc964bae None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:58.904559 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-7d135109-0f69-47dc-8739-2edadc964bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:58.918418 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d135109-0f69-47dc-8739-2edadc964bae None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:58.923348 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a92e0c0-35a4-4b23-9555-57f3fd924ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:58.924372 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92e0c0-35a4-4b23-9555-57f3fd924ed7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:58.924742 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92e0c0-35a4-4b23-9555-57f3fd924ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:58.925070 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92e0c0-35a4-4b23-9555-57f3fd924ed7 admin admin] PATH_INFO: `/v3/projects/505b81605c2e4202901ea9b0480512e8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:58.926335 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a92e0c0-35a4-4b23-9555-57f3fd924ed7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=505b81605c2e4202901ea9b0480512e8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:58.928286 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a92e0c0-35a4-4b23-9555-57f3fd924ed7 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:59.104740 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-4a92e0c0-35a4-4b23-9555-57f3fd924ed7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '505b81605c2e4202901ea9b0480512e8', 'request_id': 'req-4a92e0c0-35a4-4b23-9555-57f3fd924ed7'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:59.112291 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-4a92e0c0-35a4-4b23-9555-57f3fd924ed7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '505b81605c2e4202901ea9b0480512e8', 'request_id': 'req-4a92e0c0-35a4-4b23-9555-57f3fd924ed7'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:59.121508 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-4a92e0c0-35a4-4b23-9555-57f3fd924ed7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '505b81605c2e4202901ea9b0480512e8', 'request_id': 'req-4a92e0c0-35a4-4b23-9555-57f3fd924ed7'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:59.249254 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-4a92e0c0-35a4-4b23-9555-57f3fd924ed7 admin admin] The token cache is being invalidate because project 505b81605c2e4202901ea9b0480512e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:46:59.249550 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-4a92e0c0-35a4-4b23-9555-57f3fd924ed7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:59.250319 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-4a92e0c0-35a4-4b23-9555-57f3fd924ed7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:59.252447 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 984/1973] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:46:58 2022] DELETE /identity/v3/projects/505b81605c2e4202901ea9b0480512e8 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:46:59.256444 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1760b6e-ab3d-4b00-ac5f-e25f6fc1923e service neutron] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:59.268077 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1760b6e-ab3d-4b00-ac5f-e25f6fc1923e service neutron] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:59.433910 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1684642-9b21-455a-bd88-9c92cfef4f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:59.434973 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1684642-9b21-455a-bd88-9c92cfef4f97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:59.435363 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1684642-9b21-455a-bd88-9c92cfef4f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:59.435792 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1684642-9b21-455a-bd88-9c92cfef4f97 admin admin] PATH_INFO: `/v3/users/13c6426c31f44abdb9261d41d5b9cb9c` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:59.437470 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1684642-9b21-455a-bd88-9c92cfef4f97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13c6426c31f44abdb9261d41d5b9cb9c)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:59.440965 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1684642-9b21-455a-bd88-9c92cfef4f97 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:59.693984 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-b1684642-9b21-455a-bd88-9c92cfef4f97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13c6426c31f44abdb9261d41d5b9cb9c'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:59.782142 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-b1684642-9b21-455a-bd88-9c92cfef4f97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13c6426c31f44abdb9261d41d5b9cb9c'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:59.782826 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-b1684642-9b21-455a-bd88-9c92cfef4f97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13c6426c31f44abdb9261d41d5b9cb9c'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:59.783940 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-b1684642-9b21-455a-bd88-9c92cfef4f97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13c6426c31f44abdb9261d41d5b9cb9c'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:59.799256 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-b1684642-9b21-455a-bd88-9c92cfef4f97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13c6426c31f44abdb9261d41d5b9cb9c'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:59.825729 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 990/1974] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:46:59 2022] DELETE /identity/v3/users/13c6426c31f44abdb9261d41d5b9cb9c => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:00.068211 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:00.441572 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a92e0c0-35a4-4b23-9555-57f3fd924ed7 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:00.443995 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-4a92e0c0-35a4-4b23-9555-57f3fd924ed7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:00.460413 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a92e0c0-35a4-4b23-9555-57f3fd924ed7 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:00.556856 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e390956-45b8-4069-ac7b-d5a69353791b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:00.558251 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e390956-45b8-4069-ac7b-d5a69353791b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:00.558612 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e390956-45b8-4069-ac7b-d5a69353791b admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:00.559017 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e390956-45b8-4069-ac7b-d5a69353791b admin admin] PATH_INFO: `/v3/projects/8462c49b73fd464ba9abc47ada56d21b` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:00.560487 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e390956-45b8-4069-ac7b-d5a69353791b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8462c49b73fd464ba9abc47ada56d21b)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:00.562937 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e390956-45b8-4069-ac7b-d5a69353791b admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:00.605252 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:00.730470 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-7e390956-45b8-4069-ac7b-d5a69353791b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8462c49b73fd464ba9abc47ada56d21b', 'request_id': 'req-7e390956-45b8-4069-ac7b-d5a69353791b'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:00.745936 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-7e390956-45b8-4069-ac7b-d5a69353791b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8462c49b73fd464ba9abc47ada56d21b', 'request_id': 'req-7e390956-45b8-4069-ac7b-d5a69353791b'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:00.755151 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-7e390956-45b8-4069-ac7b-d5a69353791b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8462c49b73fd464ba9abc47ada56d21b', 'request_id': 'req-7e390956-45b8-4069-ac7b-d5a69353791b'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:00.884555 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-7e390956-45b8-4069-ac7b-d5a69353791b admin admin] The token cache is being invalidate because project 8462c49b73fd464ba9abc47ada56d21b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:47:00.884853 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-7e390956-45b8-4069-ac7b-d5a69353791b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:00.885527 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-7e390956-45b8-4069-ac7b-d5a69353791b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:00.887130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 985/1975] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:47:00 2022] DELETE /identity/v3/projects/8462c49b73fd464ba9abc47ada56d21b => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:00.891741 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1684642-9b21-455a-bd88-9c92cfef4f97 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:00.897794 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-b1684642-9b21-455a-bd88-9c92cfef4f97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:00.912462 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1684642-9b21-455a-bd88-9c92cfef4f97 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:00.994531 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=79,keystone:DELETE=18,keystone:INSERT=3 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:00.999594 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ff79d2-e1c9-47ab-a6c1-3f0800146ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:01.004642 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ff79d2-e1c9-47ab-a6c1-3f0800146ef9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:01.007424 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ff79d2-e1c9-47ab-a6c1-3f0800146ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:01.007424 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ff79d2-e1c9-47ab-a6c1-3f0800146ef9 admin admin] PATH_INFO: `/v3/users/19eec26f1b9c4ab485d3bbc90dc7327a` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:01.010131 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ff79d2-e1c9-47ab-a6c1-3f0800146ef9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19eec26f1b9c4ab485d3bbc90dc7327a)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:01.012326 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ff79d2-e1c9-47ab-a6c1-3f0800146ef9 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:01.292628 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-09ff79d2-e1c9-47ab-a6c1-3f0800146ef9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19eec26f1b9c4ab485d3bbc90dc7327a'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:01.449680 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-09ff79d2-e1c9-47ab-a6c1-3f0800146ef9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19eec26f1b9c4ab485d3bbc90dc7327a'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:01.450752 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-09ff79d2-e1c9-47ab-a6c1-3f0800146ef9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19eec26f1b9c4ab485d3bbc90dc7327a'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:01.452161 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-09ff79d2-e1c9-47ab-a6c1-3f0800146ef9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19eec26f1b9c4ab485d3bbc90dc7327a'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:01.465145 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-09ff79d2-e1c9-47ab-a6c1-3f0800146ef9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19eec26f1b9c4ab485d3bbc90dc7327a'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:01.494639 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 991/1976] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:47:00 2022] DELETE /identity/v3/users/19eec26f1b9c4ab485d3bbc90dc7327a => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:02.350464 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e390956-45b8-4069-ac7b-d5a69353791b admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:02.352004 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-7e390956-45b8-4069-ac7b-d5a69353791b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:02.379294 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e390956-45b8-4069-ac7b-d5a69353791b admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:02.518743 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00443343-87e5-49c8-a974-d5d1eeb74d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:02.519990 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00443343-87e5-49c8-a974-d5d1eeb74d39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:02.520467 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00443343-87e5-49c8-a974-d5d1eeb74d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:02.520900 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00443343-87e5-49c8-a974-d5d1eeb74d39 admin admin] PATH_INFO: `/v3/projects/daa1d75d7376465eb9ea857de06b51ab` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:02.522453 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00443343-87e5-49c8-a974-d5d1eeb74d39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=daa1d75d7376465eb9ea857de06b51ab)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:02.529380 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00443343-87e5-49c8-a974-d5d1eeb74d39 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:02.669599 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-00443343-87e5-49c8-a974-d5d1eeb74d39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'daa1d75d7376465eb9ea857de06b51ab', 'request_id': 'req-00443343-87e5-49c8-a974-d5d1eeb74d39'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:02.677191 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-00443343-87e5-49c8-a974-d5d1eeb74d39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'daa1d75d7376465eb9ea857de06b51ab', 'request_id': 'req-00443343-87e5-49c8-a974-d5d1eeb74d39'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:02.698026 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-00443343-87e5-49c8-a974-d5d1eeb74d39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'daa1d75d7376465eb9ea857de06b51ab', 'request_id': 'req-00443343-87e5-49c8-a974-d5d1eeb74d39'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:02.803847 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-00443343-87e5-49c8-a974-d5d1eeb74d39 admin admin] The token cache is being invalidate because project daa1d75d7376465eb9ea857de06b51ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:47:02.804536 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-00443343-87e5-49c8-a974-d5d1eeb74d39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:02.808110 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-00443343-87e5-49c8-a974-d5d1eeb74d39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:02.810212 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 986/1977] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:47:02 2022] DELETE /identity/v3/projects/daa1d75d7376465eb9ea857de06b51ab => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:02.816252 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd89f0b-d802-4105-bda4-88918cda161b None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:02.817282 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd89f0b-d802-4105-bda4-88918cda161b None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:02.817818 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd89f0b-d802-4105-bda4-88918cda161b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:02.842858 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-dcd89f0b-d802-4105-bda4-88918cda161b None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:02.844142 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-dcd89f0b-d802-4105-bda4-88918cda161b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:02.919644 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 992/1978] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:02.921613 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51523524-1ee4-4fe8-afb2-e88681e88c17 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:02.922156 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51523524-1ee4-4fe8-afb2-e88681e88c17 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:02.922658 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51523524-1ee4-4fe8-afb2-e88681e88c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:02.949551 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-51523524-1ee4-4fe8-afb2-e88681e88c17 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:02.950797 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-51523524-1ee4-4fe8-afb2-e88681e88c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:02.964498 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 987/1979] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:47:02.965601 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd89f0b-d802-4105-bda4-88918cda161b None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:02.989203 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd89f0b-d802-4105-bda4-88918cda161b None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:03.005451 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00feba27-4d80-4572-92e2-bae1c0a87fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:03.006512 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00feba27-4d80-4572-92e2-bae1c0a87fff admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:03.006998 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00feba27-4d80-4572-92e2-bae1c0a87fff admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:03.007435 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00feba27-4d80-4572-92e2-bae1c0a87fff admin admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:03.008418 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00feba27-4d80-4572-92e2-bae1c0a87fff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:47:03.009517 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00feba27-4d80-4572-92e2-bae1c0a87fff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:03.011829 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00feba27-4d80-4572-92e2-bae1c0a87fff admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:03.029589 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 993/1980] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 16:47:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:47:03.038294 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51523524-1ee4-4fe8-afb2-e88681e88c17 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:03.050607 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51523524-1ee4-4fe8-afb2-e88681e88c17 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:03.064398 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710e6ed3-683c-4a32-aa3d-9e824d5680e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:03.069131 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e6ed3-683c-4a32-aa3d-9e824d5680e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:03.069131 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e6ed3-683c-4a32-aa3d-9e824d5680e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:03.069131 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e6ed3-683c-4a32-aa3d-9e824d5680e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:03.069131 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710e6ed3-683c-4a32-aa3d-9e824d5680e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:03.070935 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710e6ed3-683c-4a32-aa3d-9e824d5680e6 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:03.142089 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:03.235537 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 988/1981] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:47:03 2022] POST /identity/v3/projects => generated 389 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:47:03.238349 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00feba27-4d80-4572-92e2-bae1c0a87fff admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:03.248935 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00feba27-4d80-4572-92e2-bae1c0a87fff admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:03.308684 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc1dbc4-a8b1-4e5c-af66-766697e66a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:03.309378 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc1dbc4-a8b1-4e5c-af66-766697e66a0c admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:03.309925 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc1dbc4-a8b1-4e5c-af66-766697e66a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:03.310390 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc1dbc4-a8b1-4e5c-af66-766697e66a0c admin admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:03.311105 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc1dbc4-a8b1-4e5c-af66-766697e66a0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:03.313266 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc1dbc4-a8b1-4e5c-af66-766697e66a0c admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:03.554204 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 994/1982] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:47:03 2022] POST /identity/v3/users => generated 295 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:47:03.554763 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710e6ed3-683c-4a32-aa3d-9e824d5680e6 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:03.569854 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710e6ed3-683c-4a32-aa3d-9e824d5680e6 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:03.578650 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe6b812-d7f6-4e2c-9c3e-37e5ef746486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:03.579343 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe6b812-d7f6-4e2c-9c3e-37e5ef746486 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:03.579862 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe6b812-d7f6-4e2c-9c3e-37e5ef746486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:03.580194 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe6b812-d7f6-4e2c-9c3e-37e5ef746486 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:03.581097 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe6b812-d7f6-4e2c-9c3e-37e5ef746486 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:03.582826 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe6b812-d7f6-4e2c-9c3e-37e5ef746486 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:03.594374 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 989/1983] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:03 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:47:03.598422 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc1dbc4-a8b1-4e5c-af66-766697e66a0c admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:03.609889 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc1dbc4-a8b1-4e5c-af66-766697e66a0c admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:03.616711 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a76de7a-eb3b-4ee4-9d43-5677be73073c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:03.619435 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a76de7a-eb3b-4ee4-9d43-5677be73073c admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:03.619435 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a76de7a-eb3b-4ee4-9d43-5677be73073c admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:03.619435 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a76de7a-eb3b-4ee4-9d43-5677be73073c admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:03.620729 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a76de7a-eb3b-4ee4-9d43-5677be73073c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:03.622822 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a76de7a-eb3b-4ee4-9d43-5677be73073c admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:03.636151 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 995/1984] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:03 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:47:03.640804 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe6b812-d7f6-4e2c-9c3e-37e5ef746486 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:03.653676 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe6b812-d7f6-4e2c-9c3e-37e5ef746486 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:03.661396 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53314531-6bc2-4e48-a26e-c6f4133257e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:03.662308 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53314531-6bc2-4e48-a26e-c6f4133257e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:03.662587 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53314531-6bc2-4e48-a26e-c6f4133257e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:03.663232 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53314531-6bc2-4e48-a26e-c6f4133257e8 admin admin] PATH_INFO: `/v3/projects/0d6322576b724e82b88ad58ab64be2fe/users/1bf54864a5114f1cb8cbcdca8ad96e98/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:03.690706 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53314531-6bc2-4e48-a26e-c6f4133257e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d6322576b724e82b88ad58ab64be2fe, user_id=1bf54864a5114f1cb8cbcdca8ad96e98, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:03.706102 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53314531-6bc2-4e48-a26e-c6f4133257e8 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:03.731405 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:03.870653 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 990/1985] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:47:03 2022] PUT /identity/v3/projects/0d6322576b724e82b88ad58ab64be2fe/users/1bf54864a5114f1cb8cbcdca8ad96e98/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:47:03.885136 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e82c1-6cf7-4a65-a7c5-d38d7a55c603 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:03.885611 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e82c1-6cf7-4a65-a7c5-d38d7a55c603 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:03.885866 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e82c1-6cf7-4a65-a7c5-d38d7a55c603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:03.931704 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-ec1e82c1-6cf7-4a65-a7c5-d38d7a55c603 None None] MFA Rules not processed for user `1bf54864a5114f1cb8cbcdca8ad96e98`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:03.933201 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-ec1e82c1-6cf7-4a65-a7c5-d38d7a55c603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:04.035278 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 996/1986] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:04.041530 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53314531-6bc2-4e48-a26e-c6f4133257e8 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:04.132626 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53314531-6bc2-4e48-a26e-c6f4133257e8 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:04.141296 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba25c7c3-8c84-4c53-a51c-77dfed80fe5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:04.142329 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25c7c3-8c84-4c53-a51c-77dfed80fe5a admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:04.142690 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25c7c3-8c84-4c53-a51c-77dfed80fe5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:04.143043 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25c7c3-8c84-4c53-a51c-77dfed80fe5a admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:04.143897 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba25c7c3-8c84-4c53-a51c-77dfed80fe5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:04.147288 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba25c7c3-8c84-4c53-a51c-77dfed80fe5a admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:04.341168 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 991/1987] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:47:04 2022] POST /identity/v3/projects => generated 387 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:47:04.343126 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1e82c1-6cf7-4a65-a7c5-d38d7a55c603 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:04.367878 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1e82c1-6cf7-4a65-a7c5-d38d7a55c603 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:04.467699 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474679ed-08a8-490d-a6c0-4c374635be6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:04.471616 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474679ed-08a8-490d-a6c0-4c374635be6a admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:04.471934 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474679ed-08a8-490d-a6c0-4c374635be6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:04.472216 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474679ed-08a8-490d-a6c0-4c374635be6a admin admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:04.473052 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474679ed-08a8-490d-a6c0-4c374635be6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:04.477983 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474679ed-08a8-490d-a6c0-4c374635be6a admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:04.655101 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 997/1988] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:47:04 2022] POST /identity/v3/users => generated 300 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:47:04.659309 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba25c7c3-8c84-4c53-a51c-77dfed80fe5a admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:04.673298 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba25c7c3-8c84-4c53-a51c-77dfed80fe5a admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:04.732735 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f21ea-4a07-444b-88a6-73a78bfeefe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:04.733697 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f21ea-4a07-444b-88a6-73a78bfeefe1 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:04.734156 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f21ea-4a07-444b-88a6-73a78bfeefe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:04.734593 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f21ea-4a07-444b-88a6-73a78bfeefe1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:04.736305 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581f21ea-4a07-444b-88a6-73a78bfeefe1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:04.744984 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581f21ea-4a07-444b-88a6-73a78bfeefe1 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:04.766654 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 992/1989] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:04 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:47:04.770853 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474679ed-08a8-490d-a6c0-4c374635be6a admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:04.788928 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474679ed-08a8-490d-a6c0-4c374635be6a admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:04.814395 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100d2eae-7cbb-4bdb-b46c-ba27c2f17635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:04.815463 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100d2eae-7cbb-4bdb-b46c-ba27c2f17635 admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:04.815847 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100d2eae-7cbb-4bdb-b46c-ba27c2f17635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:04.816199 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100d2eae-7cbb-4bdb-b46c-ba27c2f17635 admin admin] PATH_INFO: `/v3/projects/2adbd96a57974042a4cecf3e437a9e90/users/7d6ee94c011146d0b8582b46d41781d5/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:04.855448 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100d2eae-7cbb-4bdb-b46c-ba27c2f17635 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2adbd96a57974042a4cecf3e437a9e90, user_id=7d6ee94c011146d0b8582b46d41781d5, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:04.858769 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100d2eae-7cbb-4bdb-b46c-ba27c2f17635 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:04.938984 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f21ea-4a07-444b-88a6-73a78bfeefe1 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:04.943577 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-581f21ea-4a07-444b-88a6-73a78bfeefe1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:04.964884 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f21ea-4a07-444b-88a6-73a78bfeefe1 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:04.979147 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a62e6c-7eb6-4b2a-96a2-3f4b765f6162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:04.979147 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a62e6c-7eb6-4b2a-96a2-3f4b765f6162 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:04.979147 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a62e6c-7eb6-4b2a-96a2-3f4b765f6162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:04.979147 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a62e6c-7eb6-4b2a-96a2-3f4b765f6162 admin admin] PATH_INFO: `/v3/users/b4df0c1223e04445a64cdf931aa48baf` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:04.980920 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a62e6c-7eb6-4b2a-96a2-3f4b765f6162 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4df0c1223e04445a64cdf931aa48baf)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:04.985030 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a62e6c-7eb6-4b2a-96a2-3f4b765f6162 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:05.014763 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 998/1990] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:47:04 2022] PUT /identity/v3/projects/2adbd96a57974042a4cecf3e437a9e90/users/7d6ee94c011146d0b8582b46d41781d5/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:05.019058 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100d2eae-7cbb-4bdb-b46c-ba27c2f17635 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:05.132531 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100d2eae-7cbb-4bdb-b46c-ba27c2f17635 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:05.148676 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2724e4e-f532-4fe8-9f44-e3b9cf37748b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:05.150360 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2724e4e-f532-4fe8-9f44-e3b9cf37748b admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:05.150684 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2724e4e-f532-4fe8-9f44-e3b9cf37748b admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:05.151011 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2724e4e-f532-4fe8-9f44-e3b9cf37748b admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:05.151949 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2724e4e-f532-4fe8-9f44-e3b9cf37748b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:05.159702 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2724e4e-f532-4fe8-9f44-e3b9cf37748b admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:05.179721 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 999/1991] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:05 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:47:05.181791 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c221a34-fbc9-4c53-8063-0f7c587c7e9b None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:05.184667 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c221a34-fbc9-4c53-8063-0f7c587c7e9b None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:05.185392 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c221a34-fbc9-4c53-8063-0f7c587c7e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:05.230714 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-5c221a34-fbc9-4c53-8063-0f7c587c7e9b None None] MFA Rules not processed for user `9e337289fc9d40e5877f6f3adc8fd8ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:05.232053 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-5c221a34-fbc9-4c53-8063-0f7c587c7e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:05.311016 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-65a62e6c-7eb6-4b2a-96a2-3f4b765f6162 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4df0c1223e04445a64cdf931aa48baf'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:05.340417 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1000/1992] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:05 2022] POST /identity/v3/auth/tokens => generated 3295 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:05.341322 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c221a34-fbc9-4c53-8063-0f7c587c7e9b None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:05.360923 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c221a34-fbc9-4c53-8063-0f7c587c7e9b None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:05.371292 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dc6da5-43ab-4b13-be5a-64b5de50efa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:05.371292 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc6da5-43ab-4b13-be5a-64b5de50efa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:05.371292 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc6da5-43ab-4b13-be5a-64b5de50efa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:05.371292 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc6da5-43ab-4b13-be5a-64b5de50efa7 admin admin] PATH_INFO: `/v3/projects/2adbd96a57974042a4cecf3e437a9e90/users/7d6ee94c011146d0b8582b46d41781d5/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:05.373910 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dc6da5-43ab-4b13-be5a-64b5de50efa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2adbd96a57974042a4cecf3e437a9e90, user_id=7d6ee94c011146d0b8582b46d41781d5, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:05.377176 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dc6da5-43ab-4b13-be5a-64b5de50efa7 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:05.390536 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1001/1993] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:47:05 2022] PUT /identity/v3/projects/2adbd96a57974042a4cecf3e437a9e90/users/7d6ee94c011146d0b8582b46d41781d5/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:05.391766 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dc6da5-43ab-4b13-be5a-64b5de50efa7 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:05.400215 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-11dc6da5-43ab-4b13-be5a-64b5de50efa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:05.493995 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-65a62e6c-7eb6-4b2a-96a2-3f4b765f6162 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4df0c1223e04445a64cdf931aa48baf'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:05.504449 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-65a62e6c-7eb6-4b2a-96a2-3f4b765f6162 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4df0c1223e04445a64cdf931aa48baf'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:05.528845 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dc6da5-43ab-4b13-be5a-64b5de50efa7 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:05.531434 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-65a62e6c-7eb6-4b2a-96a2-3f4b765f6162 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4df0c1223e04445a64cdf931aa48baf'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:05.532361 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-65a62e6c-7eb6-4b2a-96a2-3f4b765f6162 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4df0c1223e04445a64cdf931aa48baf'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:05.541898 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eafc2c-0e01-40be-829f-ddaae19f813a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:05.542624 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eafc2c-0e01-40be-829f-ddaae19f813a service nova] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:05.542912 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eafc2c-0e01-40be-829f-ddaae19f813a service nova] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:05.543193 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eafc2c-0e01-40be-829f-ddaae19f813a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:05.545257 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-21eafc2c-0e01-40be-829f-ddaae19f813a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:05.547472 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 993/1994] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:47:04 2022] DELETE /identity/v3/users/b4df0c1223e04445a64cdf931aa48baf => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:47:05.553163 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56018330-4d7b-4d32-bb85-d2a5c2f9b573 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:05.553531 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56018330-4d7b-4d32-bb85-d2a5c2f9b573 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:05.554655 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56018330-4d7b-4d32-bb85-d2a5c2f9b573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:05.621501 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-56018330-4d7b-4d32-bb85-d2a5c2f9b573 None None] MFA Rules not processed for user `7d6ee94c011146d0b8582b46d41781d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:05.623218 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-56018330-4d7b-4d32-bb85-d2a5c2f9b573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:05.712217 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eafc2c-0e01-40be-829f-ddaae19f813a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:05.715189 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eafc2c-0e01-40be-829f-ddaae19f813a service nova] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:05.723985 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 994/1995] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:05 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:05.723985 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b736f99-ca09-4127-8eb5-3b5a8db7716b None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:05.723985 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b736f99-ca09-4127-8eb5-3b5a8db7716b None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:05.723985 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b736f99-ca09-4127-8eb5-3b5a8db7716b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:05.736371 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1002/1996] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:05 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:47:05.738756 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eafc2c-0e01-40be-829f-ddaae19f813a service nova] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:05.740975 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-21eafc2c-0e01-40be-829f-ddaae19f813a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:05.742259 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-1b736f99-ca09-4127-8eb5-3b5a8db7716b None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:05.743931 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-1b736f99-ca09-4127-8eb5-3b5a8db7716b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:05.857636 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 995/1997] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:05.865810 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b736f99-ca09-4127-8eb5-3b5a8db7716b None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:05.870143 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-1b736f99-ca09-4127-8eb5-3b5a8db7716b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:05.873301 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eafc2c-0e01-40be-829f-ddaae19f813a service nova] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:05.933526 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225c6d44-ed70-4685-83f9-64054223c2da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:05.934353 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c6d44-ed70-4685-83f9-64054223c2da service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:05.934465 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c6d44-ed70-4685-83f9-64054223c2da service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:05.934749 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c6d44-ed70-4685-83f9-64054223c2da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:05.944571 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225c6d44-ed70-4685-83f9-64054223c2da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:05.945216 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b736f99-ca09-4127-8eb5-3b5a8db7716b None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:05.946282 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225c6d44-ed70-4685-83f9-64054223c2da service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:05.950878 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f550fb68-a5ad-4d18-9749-ffec32469811 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:05.951913 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f550fb68-a5ad-4d18-9749-ffec32469811 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:05.953078 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f550fb68-a5ad-4d18-9749-ffec32469811 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:05.953373 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f550fb68-a5ad-4d18-9749-ffec32469811 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:05.957836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1003/1998] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:05 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:47:05.967011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f550fb68-a5ad-4d18-9749-ffec32469811 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:05.968683 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f550fb68-a5ad-4d18-9749-ffec32469811 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:05.981087 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 996/1999] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:47:05.988069 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:06.186997 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225c6d44-ed70-4685-83f9-64054223c2da service cinder] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:06.188779 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-225c6d44-ed70-4685-83f9-64054223c2da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:06.206318 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225c6d44-ed70-4685-83f9-64054223c2da service cinder] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:06.214910 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850ba1ee-3cd4-40e7-8a0e-ec0b6ec506a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:06.217551 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ba1ee-3cd4-40e7-8a0e-ec0b6ec506a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:06.217551 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ba1ee-3cd4-40e7-8a0e-ec0b6ec506a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:06.217551 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ba1ee-3cd4-40e7-8a0e-ec0b6ec506a2 admin admin] PATH_INFO: `/v3/users/1bf54864a5114f1cb8cbcdca8ad96e98` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:06.220062 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850ba1ee-3cd4-40e7-8a0e-ec0b6ec506a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bf54864a5114f1cb8cbcdca8ad96e98)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:06.222134 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850ba1ee-3cd4-40e7-8a0e-ec0b6ec506a2 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:06.456187 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f550fb68-a5ad-4d18-9749-ffec32469811 service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:06.457662 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f550fb68-a5ad-4d18-9749-ffec32469811 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:06.477284 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-850ba1ee-3cd4-40e7-8a0e-ec0b6ec506a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bf54864a5114f1cb8cbcdca8ad96e98'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:06.477867 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f550fb68-a5ad-4d18-9749-ffec32469811 service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:06.483475 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e0d234-0072-4b9f-a92e-65838133d205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:06.484421 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e0d234-0072-4b9f-a92e-65838133d205 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:06.484699 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e0d234-0072-4b9f-a92e-65838133d205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:06.484974 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e0d234-0072-4b9f-a92e-65838133d205 admin admin] PATH_INFO: `/v3/users/9d219e2dd3a14573b67858bda9eb346b` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:06.486692 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e0d234-0072-4b9f-a92e-65838133d205 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d219e2dd3a14573b67858bda9eb346b)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:06.489011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e0d234-0072-4b9f-a92e-65838133d205 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:06.563197 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-850ba1ee-3cd4-40e7-8a0e-ec0b6ec506a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bf54864a5114f1cb8cbcdca8ad96e98'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:06.563751 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-850ba1ee-3cd4-40e7-8a0e-ec0b6ec506a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bf54864a5114f1cb8cbcdca8ad96e98'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:06.564609 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-850ba1ee-3cd4-40e7-8a0e-ec0b6ec506a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bf54864a5114f1cb8cbcdca8ad96e98'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:06.580646 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-850ba1ee-3cd4-40e7-8a0e-ec0b6ec506a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bf54864a5114f1cb8cbcdca8ad96e98'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:06.609719 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1004/2000] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:47:06 2022] DELETE /identity/v3/users/1bf54864a5114f1cb8cbcdca8ad96e98 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:06.610768 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850ba1ee-3cd4-40e7-8a0e-ec0b6ec506a2 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:06.613227 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-850ba1ee-3cd4-40e7-8a0e-ec0b6ec506a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:06.699027 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850ba1ee-3cd4-40e7-8a0e-ec0b6ec506a2 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:06.703901 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7c9ad1-a5d4-4976-a582-39a184143ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:06.704907 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c9ad1-a5d4-4976-a582-39a184143ff6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:06.705302 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c9ad1-a5d4-4976-a582-39a184143ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:06.705635 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c9ad1-a5d4-4976-a582-39a184143ff6 admin admin] PATH_INFO: `/v3/projects/6017e4b8ab1e4f2fbdcf016ad3abce9f` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:06.707156 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7c9ad1-a5d4-4976-a582-39a184143ff6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6017e4b8ab1e4f2fbdcf016ad3abce9f)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:06.709206 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7c9ad1-a5d4-4976-a582-39a184143ff6 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:06.742849 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-f4e0d234-0072-4b9f-a92e-65838133d205 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d219e2dd3a14573b67858bda9eb346b'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:07.008529 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-f4e0d234-0072-4b9f-a92e-65838133d205 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d219e2dd3a14573b67858bda9eb346b'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:07.028702 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-8d7c9ad1-a5d4-4976-a582-39a184143ff6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6017e4b8ab1e4f2fbdcf016ad3abce9f', 'request_id': 'req-8d7c9ad1-a5d4-4976-a582-39a184143ff6'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:07.036543 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-f4e0d234-0072-4b9f-a92e-65838133d205 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d219e2dd3a14573b67858bda9eb346b'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:07.036865 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-8d7c9ad1-a5d4-4976-a582-39a184143ff6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6017e4b8ab1e4f2fbdcf016ad3abce9f', 'request_id': 'req-8d7c9ad1-a5d4-4976-a582-39a184143ff6'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:07.057094 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-f4e0d234-0072-4b9f-a92e-65838133d205 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d219e2dd3a14573b67858bda9eb346b'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:07.058143 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-f4e0d234-0072-4b9f-a92e-65838133d205 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d219e2dd3a14573b67858bda9eb346b'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:07.065170 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 997/2001] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:47:06 2022] DELETE /identity/v3/users/9d219e2dd3a14573b67858bda9eb346b => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:07.067103 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065ff14d-1656-4d2c-813d-135a1879394c None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:07.067513 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065ff14d-1656-4d2c-813d-135a1879394c None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:07.068041 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065ff14d-1656-4d2c-813d-135a1879394c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:07.095238 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-065ff14d-1656-4d2c-813d-135a1879394c None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:07.097479 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-065ff14d-1656-4d2c-813d-135a1879394c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:07.130187 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-8d7c9ad1-a5d4-4976-a582-39a184143ff6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6017e4b8ab1e4f2fbdcf016ad3abce9f', 'request_id': 'req-8d7c9ad1-a5d4-4976-a582-39a184143ff6'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:07.147863 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-8d7c9ad1-a5d4-4976-a582-39a184143ff6 admin admin] The token cache is being invalidate because project 6017e4b8ab1e4f2fbdcf016ad3abce9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:47:07.148210 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-8d7c9ad1-a5d4-4976-a582-39a184143ff6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:07.151462 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-8d7c9ad1-a5d4-4976-a582-39a184143ff6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:07.158882 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1005/2002] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:47:06 2022] DELETE /identity/v3/projects/6017e4b8ab1e4f2fbdcf016ad3abce9f => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:07.158882 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d26be-9ca1-443c-9344-8bd0353475d4 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:07.158882 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d26be-9ca1-443c-9344-8bd0353475d4 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:07.158882 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d26be-9ca1-443c-9344-8bd0353475d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:07.192293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 998/2003] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:07.193221 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-644d26be-9ca1-443c-9344-8bd0353475d4 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:07.193692 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065ff14d-1656-4d2c-813d-135a1879394c None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:07.194754 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-644d26be-9ca1-443c-9344-8bd0353475d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:07.196346 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-065ff14d-1656-4d2c-813d-135a1879394c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:07.211920 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1006/2004] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:07.213251 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644d26be-9ca1-443c-9344-8bd0353475d4 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:07.216234 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-644d26be-9ca1-443c-9344-8bd0353475d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:07.222923 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065ff14d-1656-4d2c-813d-135a1879394c None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:07.236217 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42cac24-ca5e-487a-adc1-76921b30c546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:07.237765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42cac24-ca5e-487a-adc1-76921b30c546 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:07.238067 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42cac24-ca5e-487a-adc1-76921b30c546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:07.238418 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42cac24-ca5e-487a-adc1-76921b30c546 admin admin] PATH_INFO: `/v3/users/3c199f3170ec44faab7464a13071092e` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:07.240189 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42cac24-ca5e-487a-adc1-76921b30c546 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c199f3170ec44faab7464a13071092e)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:07.242127 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42cac24-ca5e-487a-adc1-76921b30c546 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:07.290882 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644d26be-9ca1-443c-9344-8bd0353475d4 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:07.296043 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7addd6b-8b72-488d-8b98-7d41cd006231 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:07.297040 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7addd6b-8b72-488d-8b98-7d41cd006231 service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:07.297370 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7addd6b-8b72-488d-8b98-7d41cd006231 service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:07.297600 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7addd6b-8b72-488d-8b98-7d41cd006231 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:07.312953 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7addd6b-8b72-488d-8b98-7d41cd006231 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:07.317253 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7addd6b-8b72-488d-8b98-7d41cd006231 service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:07.339672 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1007/2005] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:47:07.342659 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7addd6b-8b72-488d-8b98-7d41cd006231 service neutron] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:07.357342 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7addd6b-8b72-488d-8b98-7d41cd006231 service neutron] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:07.367327 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab68102-5326-4748-8faf-094f1e29f5cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:07.371381 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab68102-5326-4748-8faf-094f1e29f5cc service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:07.371381 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab68102-5326-4748-8faf-094f1e29f5cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:07.371381 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab68102-5326-4748-8faf-094f1e29f5cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:07.386735 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab68102-5326-4748-8faf-094f1e29f5cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:07.388709 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab68102-5326-4748-8faf-094f1e29f5cc service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:07.402337 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1008/2006] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:47:07.534796 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-a42cac24-ca5e-487a-adc1-76921b30c546 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c199f3170ec44faab7464a13071092e'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:07.599170 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-a42cac24-ca5e-487a-adc1-76921b30c546 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c199f3170ec44faab7464a13071092e'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:07.602110 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:07.615456 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-a42cac24-ca5e-487a-adc1-76921b30c546 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c199f3170ec44faab7464a13071092e'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:07.635922 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-a42cac24-ca5e-487a-adc1-76921b30c546 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c199f3170ec44faab7464a13071092e'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:07.636749 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-a42cac24-ca5e-487a-adc1-76921b30c546 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c199f3170ec44faab7464a13071092e'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:07.643964 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 999/2007] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:47:07 2022] DELETE /identity/v3/users/3c199f3170ec44faab7464a13071092e => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:08.311888 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab68102-5326-4748-8faf-094f1e29f5cc service neutron] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:08.313630 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-2ab68102-5326-4748-8faf-094f1e29f5cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:08.331127 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab68102-5326-4748-8faf-094f1e29f5cc service neutron] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:08.397671 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42cac24-ca5e-487a-adc1-76921b30c546 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:08.399876 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a42cac24-ca5e-487a-adc1-76921b30c546 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:08.484894 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa0def-47b9-484c-b8ea-f44f3dd0c75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:08.486405 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa0def-47b9-484c-b8ea-f44f3dd0c75a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:08.489379 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa0def-47b9-484c-b8ea-f44f3dd0c75a admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:08.497645 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa0def-47b9-484c-b8ea-f44f3dd0c75a admin admin] PATH_INFO: `/v3/projects/0d6322576b724e82b88ad58ab64be2fe` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:08.501943 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aa0def-47b9-484c-b8ea-f44f3dd0c75a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d6322576b724e82b88ad58ab64be2fe)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:08.508527 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aa0def-47b9-484c-b8ea-f44f3dd0c75a admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:08.601004 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42cac24-ca5e-487a-adc1-76921b30c546 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:08.609704 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a569c04-c877-4e7f-bf7c-726c0d30d755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:08.611941 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a569c04-c877-4e7f-bf7c-726c0d30d755 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:08.616185 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a569c04-c877-4e7f-bf7c-726c0d30d755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:08.616185 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a569c04-c877-4e7f-bf7c-726c0d30d755 admin admin] PATH_INFO: `/v3/projects/536017dde63546ff99f8cfef2890dc5a` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:08.616185 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a569c04-c877-4e7f-bf7c-726c0d30d755 admin admin] RBAC: Authorizing `identity:delete_project(project_id=536017dde63546ff99f8cfef2890dc5a)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:08.616185 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a569c04-c877-4e7f-bf7c-726c0d30d755 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:08.682419 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-79aa0def-47b9-484c-b8ea-f44f3dd0c75a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d6322576b724e82b88ad58ab64be2fe', 'request_id': 'req-79aa0def-47b9-484c-b8ea-f44f3dd0c75a'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:08.699809 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-79aa0def-47b9-484c-b8ea-f44f3dd0c75a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d6322576b724e82b88ad58ab64be2fe', 'request_id': 'req-79aa0def-47b9-484c-b8ea-f44f3dd0c75a'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:08.850121 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-79aa0def-47b9-484c-b8ea-f44f3dd0c75a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d6322576b724e82b88ad58ab64be2fe', 'request_id': 'req-79aa0def-47b9-484c-b8ea-f44f3dd0c75a'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:08.864610 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-79aa0def-47b9-484c-b8ea-f44f3dd0c75a admin admin] The token cache is being invalidate because project 0d6322576b724e82b88ad58ab64be2fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:47:08.864932 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-79aa0def-47b9-484c-b8ea-f44f3dd0c75a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:08.865628 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-79aa0def-47b9-484c-b8ea-f44f3dd0c75a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:08.867587 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1009/2008] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:47:08 2022] DELETE /identity/v3/projects/0d6322576b724e82b88ad58ab64be2fe => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:08.868444 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa0def-47b9-484c-b8ea-f44f3dd0c75a admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:08.870521 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-79aa0def-47b9-484c-b8ea-f44f3dd0c75a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:08.875965 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-8a569c04-c877-4e7f-bf7c-726c0d30d755 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '536017dde63546ff99f8cfef2890dc5a', 'request_id': 'req-8a569c04-c877-4e7f-bf7c-726c0d30d755'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:08.885742 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-8a569c04-c877-4e7f-bf7c-726c0d30d755 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '536017dde63546ff99f8cfef2890dc5a', 'request_id': 'req-8a569c04-c877-4e7f-bf7c-726c0d30d755'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:08.915677 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-8a569c04-c877-4e7f-bf7c-726c0d30d755 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '536017dde63546ff99f8cfef2890dc5a', 'request_id': 'req-8a569c04-c877-4e7f-bf7c-726c0d30d755'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:09.003984 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa0def-47b9-484c-b8ea-f44f3dd0c75a admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:09.031037 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-8a569c04-c877-4e7f-bf7c-726c0d30d755 admin admin] The token cache is being invalidate because project 536017dde63546ff99f8cfef2890dc5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:47:09.031437 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-8a569c04-c877-4e7f-bf7c-726c0d30d755 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:09.032208 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-8a569c04-c877-4e7f-bf7c-726c0d30d755 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:09.041457 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1000/2009] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:47:08 2022] DELETE /identity/v3/projects/536017dde63546ff99f8cfef2890dc5a => generated 0 bytes in 641 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:47:09.041457 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a569c04-c877-4e7f-bf7c-726c0d30d755 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:09.045072 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-8a569c04-c877-4e7f-bf7c-726c0d30d755 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:09.122488 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62b342a-3eb5-417b-987a-67834b2dd367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:09.123675 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b342a-3eb5-417b-987a-67834b2dd367 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:09.123970 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b342a-3eb5-417b-987a-67834b2dd367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:09.124249 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b342a-3eb5-417b-987a-67834b2dd367 admin admin] PATH_INFO: `/v3/projects/a08689affad24ad388a204ec66fc23e5` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:09.126872 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62b342a-3eb5-417b-987a-67834b2dd367 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a08689affad24ad388a204ec66fc23e5)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:09.130490 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62b342a-3eb5-417b-987a-67834b2dd367 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:09.209841 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a569c04-c877-4e7f-bf7c-726c0d30d755 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:09.223075 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b776fd60-a82d-42ee-a6c3-b18044432c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:09.224284 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b776fd60-a82d-42ee-a6c3-b18044432c4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:09.224681 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b776fd60-a82d-42ee-a6c3-b18044432c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:09.225031 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b776fd60-a82d-42ee-a6c3-b18044432c4e admin admin] PATH_INFO: `/v3/users/7d6ee94c011146d0b8582b46d41781d5` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:09.231532 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b776fd60-a82d-42ee-a6c3-b18044432c4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d6ee94c011146d0b8582b46d41781d5)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:09.235752 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b776fd60-a82d-42ee-a6c3-b18044432c4e admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:09.443679 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-a62b342a-3eb5-417b-987a-67834b2dd367 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a08689affad24ad388a204ec66fc23e5', 'request_id': 'req-a62b342a-3eb5-417b-987a-67834b2dd367'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:09.465731 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-a62b342a-3eb5-417b-987a-67834b2dd367 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a08689affad24ad388a204ec66fc23e5', 'request_id': 'req-a62b342a-3eb5-417b-987a-67834b2dd367'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:09.554594 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b776fd60-a82d-42ee-a6c3-b18044432c4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d6ee94c011146d0b8582b46d41781d5'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:09.588107 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-a62b342a-3eb5-417b-987a-67834b2dd367 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a08689affad24ad388a204ec66fc23e5', 'request_id': 'req-a62b342a-3eb5-417b-987a-67834b2dd367'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:09.610848 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-a62b342a-3eb5-417b-987a-67834b2dd367 admin admin] The token cache is being invalidate because project a08689affad24ad388a204ec66fc23e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:47:09.611135 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-a62b342a-3eb5-417b-987a-67834b2dd367 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:09.612099 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-a62b342a-3eb5-417b-987a-67834b2dd367 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:09.627898 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1010/2010] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:47:08 2022] DELETE /identity/v3/projects/a08689affad24ad388a204ec66fc23e5 => generated 0 bytes in 760 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:09.628985 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62b342a-3eb5-417b-987a-67834b2dd367 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:09.635025 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-a62b342a-3eb5-417b-987a-67834b2dd367 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:09.707001 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:09.707901 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b776fd60-a82d-42ee-a6c3-b18044432c4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d6ee94c011146d0b8582b46d41781d5'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:09.728007 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b776fd60-a82d-42ee-a6c3-b18044432c4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d6ee94c011146d0b8582b46d41781d5'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:09.780205 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b776fd60-a82d-42ee-a6c3-b18044432c4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d6ee94c011146d0b8582b46d41781d5'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:09.782144 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b776fd60-a82d-42ee-a6c3-b18044432c4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d6ee94c011146d0b8582b46d41781d5'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:09.803781 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1001/2011] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:47:09 2022] DELETE /identity/v3/users/7d6ee94c011146d0b8582b46d41781d5 => generated 0 bytes in 760 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:09.817509 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b58970-7db2-466b-a620-1393243f0498 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:09.821113 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b58970-7db2-466b-a620-1393243f0498 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:09.824346 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b58970-7db2-466b-a620-1393243f0498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:09.893979 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62b342a-3eb5-417b-987a-67834b2dd367 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:09.911473 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-97b58970-7db2-466b-a620-1393243f0498 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:09.912771 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-97b58970-7db2-466b-a620-1393243f0498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:09.949885 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=86,keystone:INSERT=6,keystone:DELETE=8 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:09.974059 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7f35ad-5660-4d36-8a04-25246d1c5935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:09.975628 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f35ad-5660-4d36-8a04-25246d1c5935 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:09.976172 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f35ad-5660-4d36-8a04-25246d1c5935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:09.976858 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f35ad-5660-4d36-8a04-25246d1c5935 admin admin] PATH_INFO: `/v3/users/64ef62a6160e40c38c47bd91464e4a79` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:09.985089 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7f35ad-5660-4d36-8a04-25246d1c5935 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64ef62a6160e40c38c47bd91464e4a79)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:09.988275 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7f35ad-5660-4d36-8a04-25246d1c5935 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:10.075779 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1002/2012] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:10.090564 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c45f7-7517-4553-8ee2-bdfae45161d3 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:10.091458 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c45f7-7517-4553-8ee2-bdfae45161d3 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:10.091458 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c45f7-7517-4553-8ee2-bdfae45161d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:10.123916 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-195c45f7-7517-4553-8ee2-bdfae45161d3 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:10.125395 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-195c45f7-7517-4553-8ee2-bdfae45161d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:10.137290 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1003/2013] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:10.143924 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c45f7-7517-4553-8ee2-bdfae45161d3 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:10.154606 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c45f7-7517-4553-8ee2-bdfae45161d3 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:10.159660 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cf788e-97c6-4ab1-802f-bd2d232e724e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:10.160532 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf788e-97c6-4ab1-802f-bd2d232e724e admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:10.160816 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf788e-97c6-4ab1-802f-bd2d232e724e admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:10.161092 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf788e-97c6-4ab1-802f-bd2d232e724e admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:10.161855 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cf788e-97c6-4ab1-802f-bd2d232e724e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:47:10.162162 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cf788e-97c6-4ab1-802f-bd2d232e724e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:10.164087 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cf788e-97c6-4ab1-802f-bd2d232e724e admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:10.183744 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1004/2014] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 16:47:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:47:10.184768 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cf788e-97c6-4ab1-802f-bd2d232e724e admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:10.200062 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cf788e-97c6-4ab1-802f-bd2d232e724e admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:10.223845 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ac31da-d086-4460-81f5-2f62987b13f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:10.223845 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ac31da-d086-4460-81f5-2f62987b13f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:10.224332 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ac31da-d086-4460-81f5-2f62987b13f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:10.224332 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ac31da-d086-4460-81f5-2f62987b13f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:10.225178 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ac31da-d086-4460-81f5-2f62987b13f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:10.228817 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ac31da-d086-4460-81f5-2f62987b13f2 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:10.282267 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-da7f35ad-5660-4d36-8a04-25246d1c5935 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64ef62a6160e40c38c47bd91464e4a79'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:10.374336 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-da7f35ad-5660-4d36-8a04-25246d1c5935 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64ef62a6160e40c38c47bd91464e4a79'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:10.375253 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-da7f35ad-5660-4d36-8a04-25246d1c5935 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64ef62a6160e40c38c47bd91464e4a79'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:10.376835 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-da7f35ad-5660-4d36-8a04-25246d1c5935 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64ef62a6160e40c38c47bd91464e4a79'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:10.389881 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1005/2015] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:47:10 2022] POST /identity/v3/projects => generated 385 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:47:10.392396 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-da7f35ad-5660-4d36-8a04-25246d1c5935 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64ef62a6160e40c38c47bd91464e4a79'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:10.394424 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ac31da-d086-4460-81f5-2f62987b13f2 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:10.396285 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-29ac31da-d086-4460-81f5-2f62987b13f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:10.435836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1011/2016] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:47:09 2022] DELETE /identity/v3/users/64ef62a6160e40c38c47bd91464e4a79 => generated 0 bytes in 807 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:10.500639 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ac31da-d086-4460-81f5-2f62987b13f2 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:10.567764 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7f35ad-5660-4d36-8a04-25246d1c5935 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:10.573058 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-da7f35ad-5660-4d36-8a04-25246d1c5935 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:10.582802 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb29a45-fbbd-4fb4-8aa1-b5e3dd092466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:10.584289 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb29a45-fbbd-4fb4-8aa1-b5e3dd092466 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:10.584779 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb29a45-fbbd-4fb4-8aa1-b5e3dd092466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:10.585267 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb29a45-fbbd-4fb4-8aa1-b5e3dd092466 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:10.587574 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb29a45-fbbd-4fb4-8aa1-b5e3dd092466 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:10.590107 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb29a45-fbbd-4fb4-8aa1-b5e3dd092466 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:10.613070 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7f35ad-5660-4d36-8a04-25246d1c5935 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:10.622048 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97bde33-6805-460b-a9e2-bf415e906e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:10.625238 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97bde33-6805-460b-a9e2-bf415e906e58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:10.625238 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97bde33-6805-460b-a9e2-bf415e906e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:10.625238 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97bde33-6805-460b-a9e2-bf415e906e58 admin admin] PATH_INFO: `/v3/projects/2adbd96a57974042a4cecf3e437a9e90` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:10.627034 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97bde33-6805-460b-a9e2-bf415e906e58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2adbd96a57974042a4cecf3e437a9e90)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:10.630223 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97bde33-6805-460b-a9e2-bf415e906e58 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:10.699239 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1006/2017] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:47:10 2022] POST /identity/v3/users => generated 293 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:47:10.722135 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb29a45-fbbd-4fb4-8aa1-b5e3dd092466 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:10.757078 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb29a45-fbbd-4fb4-8aa1-b5e3dd092466 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:10.796056 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-b97bde33-6805-460b-a9e2-bf415e906e58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2adbd96a57974042a4cecf3e437a9e90', 'request_id': 'req-b97bde33-6805-460b-a9e2-bf415e906e58'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:10.811778 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-b97bde33-6805-460b-a9e2-bf415e906e58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2adbd96a57974042a4cecf3e437a9e90', 'request_id': 'req-b97bde33-6805-460b-a9e2-bf415e906e58'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:10.896884 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-b97bde33-6805-460b-a9e2-bf415e906e58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2adbd96a57974042a4cecf3e437a9e90', 'request_id': 'req-b97bde33-6805-460b-a9e2-bf415e906e58'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:10.914737 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-b97bde33-6805-460b-a9e2-bf415e906e58 admin admin] The token cache is being invalidate because project 2adbd96a57974042a4cecf3e437a9e90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:47:10.915088 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-b97bde33-6805-460b-a9e2-bf415e906e58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:10.916206 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-b97bde33-6805-460b-a9e2-bf415e906e58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:10.922331 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1012/2018] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:47:10 2022] DELETE /identity/v3/projects/2adbd96a57974042a4cecf3e437a9e90 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:10.922510 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c6b7cb-003c-43cb-b12e-58f3a3fc3cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:10.922510 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6b7cb-003c-43cb-b12e-58f3a3fc3cc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:10.922510 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6b7cb-003c-43cb-b12e-58f3a3fc3cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:10.922510 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6b7cb-003c-43cb-b12e-58f3a3fc3cc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:10.922946 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c6b7cb-003c-43cb-b12e-58f3a3fc3cc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:10.924443 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c6b7cb-003c-43cb-b12e-58f3a3fc3cc0 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:10.924914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54528b8f-7052-40d1-b912-d3757ff65702 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:10.925205 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54528b8f-7052-40d1-b912-d3757ff65702 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:10.925484 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54528b8f-7052-40d1-b912-d3757ff65702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:10.944779 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1007/2019] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:10 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:47:10.951408 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c6b7cb-003c-43cb-b12e-58f3a3fc3cc0 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:10.954158 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-72c6b7cb-003c-43cb-b12e-58f3a3fc3cc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:10.971992 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-54528b8f-7052-40d1-b912-d3757ff65702 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:10.975428 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-54528b8f-7052-40d1-b912-d3757ff65702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:10.984220 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c6b7cb-003c-43cb-b12e-58f3a3fc3cc0 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:10.995013 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c108f-352c-45a2-a5d2-1acfd841139d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:10.999665 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c108f-352c-45a2-a5d2-1acfd841139d admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:11.002204 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1013/2020] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:11.004341 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c108f-352c-45a2-a5d2-1acfd841139d admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:11.004680 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c108f-352c-45a2-a5d2-1acfd841139d admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:11.005562 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8c108f-352c-45a2-a5d2-1acfd841139d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:11.007308 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8c108f-352c-45a2-a5d2-1acfd841139d admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:11.011389 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b409e0-9b53-4cdf-ab8f-63b98d88d14e None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:11.023182 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b409e0-9b53-4cdf-ab8f-63b98d88d14e None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:11.023182 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b409e0-9b53-4cdf-ab8f-63b98d88d14e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:11.031105 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1008/2021] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:10 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:47:11.034301 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c108f-352c-45a2-a5d2-1acfd841139d admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:11.058433 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c108f-352c-45a2-a5d2-1acfd841139d admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:11.064652 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-60b409e0-9b53-4cdf-ab8f-63b98d88d14e None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:11.065005 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef2cfff-f04f-4fb6-a5b1-543457f18a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:11.066024 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef2cfff-f04f-4fb6-a5b1-543457f18a63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:11.066310 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef2cfff-f04f-4fb6-a5b1-543457f18a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:11.072143 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef2cfff-f04f-4fb6-a5b1-543457f18a63 admin admin] PATH_INFO: `/v3/projects/5775427a9b2e47fd82f56c6e7a76169b/users/4d312024dc2e40ca964f4d009ec87d00/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:11.072292 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-60b409e0-9b53-4cdf-ab8f-63b98d88d14e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:11.082649 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1014/2022] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:47:11.084601 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c26d1-bac6-4b86-a8ae-57dd29014456 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:11.085126 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c26d1-bac6-4b86-a8ae-57dd29014456 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:11.085626 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c26d1-bac6-4b86-a8ae-57dd29014456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:11.111579 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef2cfff-f04f-4fb6-a5b1-543457f18a63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5775427a9b2e47fd82f56c6e7a76169b, user_id=4d312024dc2e40ca964f4d009ec87d00, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:11.114342 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef2cfff-f04f-4fb6-a5b1-543457f18a63 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:11.116013 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-3c6c26d1-bac6-4b86-a8ae-57dd29014456 None None] MFA Rules not processed for user `6472100830a54d6a8e21ee55b68e6a02`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:11.117777 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-3c6c26d1-bac6-4b86-a8ae-57dd29014456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:11.241077 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1009/2023] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:47:11 2022] PUT /identity/v3/projects/5775427a9b2e47fd82f56c6e7a76169b/users/4d312024dc2e40ca964f4d009ec87d00/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:47:11.242103 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef2cfff-f04f-4fb6-a5b1-543457f18a63 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:11.253998 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1015/2024] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:11.255952 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c26d1-bac6-4b86-a8ae-57dd29014456 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:11.258428 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-3c6c26d1-bac6-4b86-a8ae-57dd29014456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:11.279054 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c26d1-bac6-4b86-a8ae-57dd29014456 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:11.399404 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef2cfff-f04f-4fb6-a5b1-543457f18a63 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:11.409792 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf86ca5-0977-4535-8d9a-968cbe0f3558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:11.410483 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf86ca5-0977-4535-8d9a-968cbe0f3558 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:11.410768 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf86ca5-0977-4535-8d9a-968cbe0f3558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:11.411142 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf86ca5-0977-4535-8d9a-968cbe0f3558 admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:11.412111 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf86ca5-0977-4535-8d9a-968cbe0f3558 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:47:11.412445 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf86ca5-0977-4535-8d9a-968cbe0f3558 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:11.418353 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf86ca5-0977-4535-8d9a-968cbe0f3558 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:11.421952 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04865fcc-ea7e-407e-a591-3605b36e5e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:11.423157 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04865fcc-ea7e-407e-a591-3605b36e5e02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:11.423715 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04865fcc-ea7e-407e-a591-3605b36e5e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:11.424194 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04865fcc-ea7e-407e-a591-3605b36e5e02 admin admin] PATH_INFO: `/v3/projects/2dbe16bd89994b56a632e6f69f49e001` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:11.425680 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04865fcc-ea7e-407e-a591-3605b36e5e02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2dbe16bd89994b56a632e6f69f49e001)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:11.428005 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04865fcc-ea7e-407e-a591-3605b36e5e02 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:11.435347 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1010/2025] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 16:47:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:47:11.438235 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db139c-0e10-40d3-a56f-244c2b5147a4 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:11.439026 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db139c-0e10-40d3-a56f-244c2b5147a4 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:11.439471 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db139c-0e10-40d3-a56f-244c2b5147a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:11.472838 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-58db139c-0e10-40d3-a56f-244c2b5147a4 None None] MFA Rules not processed for user `4d312024dc2e40ca964f4d009ec87d00`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:11.474070 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-58db139c-0e10-40d3-a56f-244c2b5147a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:11.603392 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1011/2026] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:11.603392 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58db139c-0e10-40d3-a56f-244c2b5147a4 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:11.624328 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-04865fcc-ea7e-407e-a591-3605b36e5e02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2dbe16bd89994b56a632e6f69f49e001', 'request_id': 'req-04865fcc-ea7e-407e-a591-3605b36e5e02'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:11.628742 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58db139c-0e10-40d3-a56f-244c2b5147a4 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:11.644405 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-04865fcc-ea7e-407e-a591-3605b36e5e02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2dbe16bd89994b56a632e6f69f49e001', 'request_id': 'req-04865fcc-ea7e-407e-a591-3605b36e5e02'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:11.721024 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8bb9fb-a969-4887-b8b2-4f422fad8720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:11.721711 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8bb9fb-a969-4887-b8b2-4f422fad8720 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:11.721998 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8bb9fb-a969-4887-b8b2-4f422fad8720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:11.722458 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8bb9fb-a969-4887-b8b2-4f422fad8720 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:11.723254 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8bb9fb-a969-4887-b8b2-4f422fad8720 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:11.725817 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8bb9fb-a969-4887-b8b2-4f422fad8720 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:11.765180 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-04865fcc-ea7e-407e-a591-3605b36e5e02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2dbe16bd89994b56a632e6f69f49e001', 'request_id': 'req-04865fcc-ea7e-407e-a591-3605b36e5e02'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:11.781253 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-04865fcc-ea7e-407e-a591-3605b36e5e02 admin admin] The token cache is being invalidate because project 2dbe16bd89994b56a632e6f69f49e001 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:47:11.781548 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-04865fcc-ea7e-407e-a591-3605b36e5e02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:11.782325 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-04865fcc-ea7e-407e-a591-3605b36e5e02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:11.784217 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1016/2027] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:47:11 2022] DELETE /identity/v3/projects/2dbe16bd89994b56a632e6f69f49e001 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:11.785144 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04865fcc-ea7e-407e-a591-3605b36e5e02 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:11.789118 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-04865fcc-ea7e-407e-a591-3605b36e5e02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:11.854492 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1012/2028] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:47:11 2022] POST /identity/v3/projects => generated 373 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:47:11.855413 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc0e32b-262d-4694-a099-e0b888c2c3b5 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:11.856131 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc0e32b-262d-4694-a099-e0b888c2c3b5 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:11.856538 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc0e32b-262d-4694-a099-e0b888c2c3b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:11.893223 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-bbc0e32b-262d-4694-a099-e0b888c2c3b5 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:11.894560 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-bbc0e32b-262d-4694-a099-e0b888c2c3b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:11.896124 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04865fcc-ea7e-407e-a591-3605b36e5e02 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:11.957431 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2519931-8a69-4ae9-9a2b-5e1fc9b2c6b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:11.958105 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2519931-8a69-4ae9-9a2b-5e1fc9b2c6b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:11.958476 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2519931-8a69-4ae9-9a2b-5e1fc9b2c6b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:11.958756 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2519931-8a69-4ae9-9a2b-5e1fc9b2c6b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:11.960575 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-b2519931-8a69-4ae9-9a2b-5e1fc9b2c6b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:11.967080 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1013/2029] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:11.974540 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc0e32b-262d-4694-a099-e0b888c2c3b5 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:11.974540 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-bbc0e32b-262d-4694-a099-e0b888c2c3b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:11.984920 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc0e32b-262d-4694-a099-e0b888c2c3b5 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:11.993043 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502fcc7c-dc43-47a1-a440-51797603ace7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:11.994818 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502fcc7c-dc43-47a1-a440-51797603ace7 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:11.995694 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502fcc7c-dc43-47a1-a440-51797603ace7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:11.996164 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502fcc7c-dc43-47a1-a440-51797603ace7 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:11.996961 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502fcc7c-dc43-47a1-a440-51797603ace7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:11.999155 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502fcc7c-dc43-47a1-a440-51797603ace7 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:12.029004 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2519931-8a69-4ae9-9a2b-5e1fc9b2c6b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:12.030466 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2519931-8a69-4ae9-9a2b-5e1fc9b2c6b5 service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:12.039438 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1017/2030] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:47:12.040759 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598dcd59-a0ba-43fa-93ab-cf48e9f3367e None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:12.041055 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598dcd59-a0ba-43fa-93ab-cf48e9f3367e None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:12.041338 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598dcd59-a0ba-43fa-93ab-cf48e9f3367e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:12.061922 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-598dcd59-a0ba-43fa-93ab-cf48e9f3367e None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:12.063092 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-598dcd59-a0ba-43fa-93ab-cf48e9f3367e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:12.072156 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1018/2031] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:12.073258 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598dcd59-a0ba-43fa-93ab-cf48e9f3367e None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:12.074615 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-598dcd59-a0ba-43fa-93ab-cf48e9f3367e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:12.086998 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598dcd59-a0ba-43fa-93ab-cf48e9f3367e None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:12.095318 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3db8a-6611-427b-be71-74f095c6397e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:12.096444 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3db8a-6611-427b-be71-74f095c6397e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:12.096519 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3db8a-6611-427b-be71-74f095c6397e admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:12.096832 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3db8a-6611-427b-be71-74f095c6397e admin admin] PATH_INFO: `/v3/users/4d312024dc2e40ca964f4d009ec87d00` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:12.098876 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc3db8a-6611-427b-be71-74f095c6397e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d312024dc2e40ca964f4d009ec87d00)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:12.105671 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc3db8a-6611-427b-be71-74f095c6397e admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:12.138122 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1014/2032] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:47:11 2022] POST /identity/v3/users => generated 287 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:47:12.139134 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502fcc7c-dc43-47a1-a440-51797603ace7 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:12.151155 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502fcc7c-dc43-47a1-a440-51797603ace7 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:12.157004 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156ec7a8-94a3-4cbc-a31a-a13a844d9e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:12.158202 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156ec7a8-94a3-4cbc-a31a-a13a844d9e64 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:12.158202 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156ec7a8-94a3-4cbc-a31a-a13a844d9e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:12.158376 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156ec7a8-94a3-4cbc-a31a-a13a844d9e64 admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:12.162643 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156ec7a8-94a3-4cbc-a31a-a13a844d9e64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:47:12.162643 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156ec7a8-94a3-4cbc-a31a-a13a844d9e64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:12.162643 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156ec7a8-94a3-4cbc-a31a-a13a844d9e64 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:12.186938 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1015/2033] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 16:47:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:47:12.191345 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156ec7a8-94a3-4cbc-a31a-a13a844d9e64 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:12.200559 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156ec7a8-94a3-4cbc-a31a-a13a844d9e64 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:12.206495 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003f00ec-d8ee-42cb-b6e7-04084af2779f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:12.206684 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f00ec-d8ee-42cb-b6e7-04084af2779f admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:12.209142 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f00ec-d8ee-42cb-b6e7-04084af2779f admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:12.209142 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f00ec-d8ee-42cb-b6e7-04084af2779f admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:12.209142 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003f00ec-d8ee-42cb-b6e7-04084af2779f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:12.210298 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003f00ec-d8ee-42cb-b6e7-04084af2779f admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:12.240499 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1016/2034] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:12 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:47:12.240499 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003f00ec-d8ee-42cb-b6e7-04084af2779f admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:12.256575 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003f00ec-d8ee-42cb-b6e7-04084af2779f admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:12.263356 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eff87c7-2850-4404-975a-eb79f83e9f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:12.264238 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff87c7-2850-4404-975a-eb79f83e9f42 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:12.264596 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff87c7-2850-4404-975a-eb79f83e9f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:12.264945 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff87c7-2850-4404-975a-eb79f83e9f42 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:12.265736 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eff87c7-2850-4404-975a-eb79f83e9f42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:12.268036 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eff87c7-2850-4404-975a-eb79f83e9f42 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:12.364953 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-5fc3db8a-6611-427b-be71-74f095c6397e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d312024dc2e40ca964f4d009ec87d00'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:12.411111 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1017/2035] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:47:12 2022] POST /identity/v3/projects => generated 377 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:47:12.412412 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eff87c7-2850-4404-975a-eb79f83e9f42 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:12.462000 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-5fc3db8a-6611-427b-be71-74f095c6397e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d312024dc2e40ca964f4d009ec87d00'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:12.463057 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-5fc3db8a-6611-427b-be71-74f095c6397e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d312024dc2e40ca964f4d009ec87d00'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:12.464066 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-5fc3db8a-6611-427b-be71-74f095c6397e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d312024dc2e40ca964f4d009ec87d00'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:12.474403 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-5fc3db8a-6611-427b-be71-74f095c6397e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d312024dc2e40ca964f4d009ec87d00'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:12.504733 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eff87c7-2850-4404-975a-eb79f83e9f42 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:12.519693 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30545900-6d70-4afc-9e80-058d3f3f3928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:12.519693 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30545900-6d70-4afc-9e80-058d3f3f3928 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:12.519693 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30545900-6d70-4afc-9e80-058d3f3f3928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:12.519693 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30545900-6d70-4afc-9e80-058d3f3f3928 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:12.520042 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30545900-6d70-4afc-9e80-058d3f3f3928 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:12.523075 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30545900-6d70-4afc-9e80-058d3f3f3928 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:12.527789 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1019/2036] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:47:12 2022] DELETE /identity/v3/users/4d312024dc2e40ca964f4d009ec87d00 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:12.529237 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3db8a-6611-427b-be71-74f095c6397e admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:12.537053 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1018/2037] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:12 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:47:12.537230 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-5fc3db8a-6611-427b-be71-74f095c6397e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:12.542044 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e8f4a8-f7ed-42cb-9b23-adf03b26479d None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:12.542611 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e8f4a8-f7ed-42cb-9b23-adf03b26479d None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:12.543119 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e8f4a8-f7ed-42cb-9b23-adf03b26479d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:12.566291 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-90e8f4a8-f7ed-42cb-9b23-adf03b26479d None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:12.569170 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-90e8f4a8-f7ed-42cb-9b23-adf03b26479d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:12.669180 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1019/2038] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:12.669794 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3db8a-6611-427b-be71-74f095c6397e admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:12.671145 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e8f4a8-f7ed-42cb-9b23-adf03b26479d None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:12.683704 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-90e8f4a8-f7ed-42cb-9b23-adf03b26479d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:12.690430 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ac9f92-933c-4080-b0ef-1e64b51cbede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:12.691140 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac9f92-933c-4080-b0ef-1e64b51cbede admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:12.691425 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac9f92-933c-4080-b0ef-1e64b51cbede admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:12.691743 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac9f92-933c-4080-b0ef-1e64b51cbede admin admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:12.692860 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ac9f92-933c-4080-b0ef-1e64b51cbede admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:12.696028 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ac9f92-933c-4080-b0ef-1e64b51cbede admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:12.698352 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e8f4a8-f7ed-42cb-9b23-adf03b26479d None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:12.710954 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c753a146-6b50-4865-b9d3-44f663a22f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:12.712998 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753a146-6b50-4865-b9d3-44f663a22f94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:12.712998 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753a146-6b50-4865-b9d3-44f663a22f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:12.713314 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753a146-6b50-4865-b9d3-44f663a22f94 admin admin] PATH_INFO: `/v3/projects/d9a814bcc8ed45c5bed3a0f5fa5531e0/users/71ffbe5c91a843118cb1796cf6960b13/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:12.730474 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c753a146-6b50-4865-b9d3-44f663a22f94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9a814bcc8ed45c5bed3a0f5fa5531e0, user_id=71ffbe5c91a843118cb1796cf6960b13, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:12.733293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c753a146-6b50-4865-b9d3-44f663a22f94 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:12.834384 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1020/2039] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:47:12 2022] PUT /identity/v3/projects/d9a814bcc8ed45c5bed3a0f5fa5531e0/users/71ffbe5c91a843118cb1796cf6960b13/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:12.839410 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c753a146-6b50-4865-b9d3-44f663a22f94 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:12.840965 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1020/2040] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:47:12 2022] POST /identity/v3/users => generated 289 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:47:12.845400 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-c753a146-6b50-4865-b9d3-44f663a22f94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:12.846156 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe5879-44a8-484d-9987-8bd153889fbb None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:12.846536 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe5879-44a8-484d-9987-8bd153889fbb None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:12.847107 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe5879-44a8-484d-9987-8bd153889fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:12.887210 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-78fe5879-44a8-484d-9987-8bd153889fbb None None] MFA Rules not processed for user `71ffbe5c91a843118cb1796cf6960b13`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:12.888509 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-78fe5879-44a8-484d-9987-8bd153889fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:12.935900 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c753a146-6b50-4865-b9d3-44f663a22f94 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:12.940300 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950b9f3a-4efc-4585-836b-c2d1b9cd3d47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:12.940964 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950b9f3a-4efc-4585-836b-c2d1b9cd3d47 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:12.941269 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950b9f3a-4efc-4585-836b-c2d1b9cd3d47 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:12.941518 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950b9f3a-4efc-4585-836b-c2d1b9cd3d47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:13.004362 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1021/2041] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:13.005318 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe5879-44a8-484d-9987-8bd153889fbb None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:13.018470 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950b9f3a-4efc-4585-836b-c2d1b9cd3d47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:13.020323 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950b9f3a-4efc-4585-836b-c2d1b9cd3d47 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:13.034861 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1021/2042] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:47:13.036084 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950b9f3a-4efc-4585-836b-c2d1b9cd3d47 service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:13.038197 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-950b9f3a-4efc-4585-836b-c2d1b9cd3d47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:13.128041 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe5879-44a8-484d-9987-8bd153889fbb None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:13.135845 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168e2de-a238-4a82-9527-96510f8d533e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:13.136148 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950b9f3a-4efc-4585-836b-c2d1b9cd3d47 service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:13.136547 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168e2de-a238-4a82-9527-96510f8d533e admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:13.136828 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168e2de-a238-4a82-9527-96510f8d533e admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:13.137102 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168e2de-a238-4a82-9527-96510f8d533e admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:13.137850 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4168e2de-a238-4a82-9527-96510f8d533e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:13.139475 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4168e2de-a238-4a82-9527-96510f8d533e admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:13.142816 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5e7be7-b4bc-4c40-9e1c-6a26cb0deb31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:13.143683 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e7be7-b4bc-4c40-9e1c-6a26cb0deb31 service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:13.144304 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e7be7-b4bc-4c40-9e1c-6a26cb0deb31 service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:13.144893 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e7be7-b4bc-4c40-9e1c-6a26cb0deb31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:13.156649 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5e7be7-b4bc-4c40-9e1c-6a26cb0deb31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:13.158751 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5e7be7-b4bc-4c40-9e1c-6a26cb0deb31 service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:13.159868 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1022/2043] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:47:13.160936 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168e2de-a238-4a82-9527-96510f8d533e admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:13.178464 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168e2de-a238-4a82-9527-96510f8d533e admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:13.178724 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1022/2044] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:13 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:47:13.192230 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b463524-14e9-4413-8ca6-1f2d9e360738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:13.193423 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b463524-14e9-4413-8ca6-1f2d9e360738 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:13.193423 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b463524-14e9-4413-8ca6-1f2d9e360738 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:13.193611 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b463524-14e9-4413-8ca6-1f2d9e360738 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:13.194222 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b463524-14e9-4413-8ca6-1f2d9e360738 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:13.196020 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b463524-14e9-4413-8ca6-1f2d9e360738 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:13.201568 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5e7be7-b4bc-4c40-9e1c-6a26cb0deb31 service cinder] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:13.205544 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:13.211971 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1023/2045] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:47:13.218056 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b463524-14e9-4413-8ca6-1f2d9e360738 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:13.218801 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5e7be7-b4bc-4c40-9e1c-6a26cb0deb31 service cinder] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:13.225451 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b7d2b9-bd22-44a9-8ba4-3fc8245ab113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:13.226575 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b7d2b9-bd22-44a9-8ba4-3fc8245ab113 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:13.227015 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b7d2b9-bd22-44a9-8ba4-3fc8245ab113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:13.227563 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b7d2b9-bd22-44a9-8ba4-3fc8245ab113 admin admin] PATH_INFO: `/v3/users/71ffbe5c91a843118cb1796cf6960b13` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:13.229836 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b7d2b9-bd22-44a9-8ba4-3fc8245ab113 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71ffbe5c91a843118cb1796cf6960b13)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:13.236283 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b7d2b9-bd22-44a9-8ba4-3fc8245ab113 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:13.236520 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b463524-14e9-4413-8ca6-1f2d9e360738 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:13.243879 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc0692-ea2a-4afb-a509-a7c3970b599f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:13.247251 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc0692-ea2a-4afb-a509-a7c3970b599f admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:13.247647 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc0692-ea2a-4afb-a509-a7c3970b599f admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:13.248127 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc0692-ea2a-4afb-a509-a7c3970b599f admin admin] PATH_INFO: `/v3/projects/2866a5dbf6a64a47839347deb8641c22/users/7389e41d736c4283a8ef8ee074a512e7/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:13.271989 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cc0692-ea2a-4afb-a509-a7c3970b599f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2866a5dbf6a64a47839347deb8641c22, user_id=7389e41d736c4283a8ef8ee074a512e7, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:13.275576 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cc0692-ea2a-4afb-a509-a7c3970b599f admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:13.461788 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1024/2046] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:47:13 2022] PUT /identity/v3/projects/2866a5dbf6a64a47839347deb8641c22/users/7389e41d736c4283a8ef8ee074a512e7/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:13.473833 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0a15b-4b7a-42b8-aff9-d2bebd1197ee None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:13.474775 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0a15b-4b7a-42b8-aff9-d2bebd1197ee None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:13.475073 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0a15b-4b7a-42b8-aff9-d2bebd1197ee None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:13.487785 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1025/2047] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jul 18 16:47:13 2022] GET /identity => generated 270 bytes in 20 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:47:13.487785 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bde785-bdcc-412a-b541-3ae2b5563f1b None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:13.487785 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bde785-bdcc-412a-b541-3ae2b5563f1b None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:13.487785 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bde785-bdcc-412a-b541-3ae2b5563f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:13.518389 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-e0bde785-bdcc-412a-b541-3ae2b5563f1b None None] MFA Rules not processed for user `7389e41d736c4283a8ef8ee074a512e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:13.522285 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-e0bde785-bdcc-412a-b541-3ae2b5563f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:13.578663 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:13.591275 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-f7b7d2b9-bd22-44a9-8ba4-3fc8245ab113 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71ffbe5c91a843118cb1796cf6960b13'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:13.629398 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1026/2048] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:13.631620 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be1cef4-0bd1-49fa-99e3-3762df63086f None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:13.632194 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be1cef4-0bd1-49fa-99e3-3762df63086f None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:13.632697 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be1cef4-0bd1-49fa-99e3-3762df63086f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:13.657802 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-8be1cef4-0bd1-49fa-99e3-3762df63086f None None] MFA Rules not processed for user `70f704a231a5474e8067a6c2dd88b355`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:13.659974 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-8be1cef4-0bd1-49fa-99e3-3762df63086f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:13.681665 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-f7b7d2b9-bd22-44a9-8ba4-3fc8245ab113 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71ffbe5c91a843118cb1796cf6960b13'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:13.694868 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-f7b7d2b9-bd22-44a9-8ba4-3fc8245ab113 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71ffbe5c91a843118cb1796cf6960b13'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:13.708509 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-f7b7d2b9-bd22-44a9-8ba4-3fc8245ab113 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71ffbe5c91a843118cb1796cf6960b13'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:13.722515 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-f7b7d2b9-bd22-44a9-8ba4-3fc8245ab113 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71ffbe5c91a843118cb1796cf6960b13'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:13.732050 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1023/2049] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:47:13 2022] DELETE /identity/v3/users/71ffbe5c91a843118cb1796cf6960b13 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:13.734303 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b7d2b9-bd22-44a9-8ba4-3fc8245ab113 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:13.739664 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f7b7d2b9-bd22-44a9-8ba4-3fc8245ab113 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:13.768155 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1027/2050] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jul 18 16:47:13 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:13.769455 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be1cef4-0bd1-49fa-99e3-3762df63086f None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:13.770945 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-8be1cef4-0bd1-49fa-99e3-3762df63086f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:13.841715 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b7d2b9-bd22-44a9-8ba4-3fc8245ab113 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:13.847946 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65950656-26ae-4f99-a043-e4a16899f78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:13.849151 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65950656-26ae-4f99-a043-e4a16899f78d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:13.849287 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65950656-26ae-4f99-a043-e4a16899f78d admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:13.849584 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65950656-26ae-4f99-a043-e4a16899f78d admin admin] PATH_INFO: `/v3/users/87094997b3b94ebea3de60446795fb7a` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:13.859297 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65950656-26ae-4f99-a043-e4a16899f78d admin admin] RBAC: Authorizing `identity:delete_user(user_id=87094997b3b94ebea3de60446795fb7a)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:13.862929 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65950656-26ae-4f99-a043-e4a16899f78d admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:13.880570 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be1cef4-0bd1-49fa-99e3-3762df63086f None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:13.886406 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1125b05c-261e-465f-9a74-da9369858629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:13.888606 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1125b05c-261e-465f-9a74-da9369858629 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:13.888606 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1125b05c-261e-465f-9a74-da9369858629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:13.888606 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1125b05c-261e-465f-9a74-da9369858629 admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:13.888606 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1125b05c-261e-465f-9a74-da9369858629 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:13.891548 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1125b05c-261e-465f-9a74-da9369858629 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:14.016125 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1028/2051] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:47:13 2022] POST /identity/v3/projects => generated 377 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:47:14.016988 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1125b05c-261e-465f-9a74-da9369858629 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:14.019006 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-1125b05c-261e-465f-9a74-da9369858629 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:14.107043 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1125b05c-261e-465f-9a74-da9369858629 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:14.112926 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb79ffe-42d1-49c8-b76c-1a06d7dcaaf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:14.113829 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb79ffe-42d1-49c8-b76c-1a06d7dcaaf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:14.114115 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb79ffe-42d1-49c8-b76c-1a06d7dcaaf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:14.114398 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb79ffe-42d1-49c8-b76c-1a06d7dcaaf9 admin admin] PATH_INFO: `/v3/projects/5775427a9b2e47fd82f56c6e7a76169b` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:14.116246 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb79ffe-42d1-49c8-b76c-1a06d7dcaaf9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5775427a9b2e47fd82f56c6e7a76169b)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:14.119592 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb79ffe-42d1-49c8-b76c-1a06d7dcaaf9 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:14.145690 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-65950656-26ae-4f99-a043-e4a16899f78d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87094997b3b94ebea3de60446795fb7a'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:14.241665 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-65950656-26ae-4f99-a043-e4a16899f78d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87094997b3b94ebea3de60446795fb7a'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:14.252385 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-65950656-26ae-4f99-a043-e4a16899f78d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87094997b3b94ebea3de60446795fb7a'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:14.268511 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-65950656-26ae-4f99-a043-e4a16899f78d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87094997b3b94ebea3de60446795fb7a'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:14.283755 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-65950656-26ae-4f99-a043-e4a16899f78d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87094997b3b94ebea3de60446795fb7a'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:14.283755 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1024/2052] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:47:13 2022] DELETE /identity/v3/users/87094997b3b94ebea3de60446795fb7a => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:14.283755 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc96931-33e0-4f8c-b868-9627994bb7c3 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:14.283755 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc96931-33e0-4f8c-b868-9627994bb7c3 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:14.283755 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc96931-33e0-4f8c-b868-9627994bb7c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:14.308996 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-edc96931-33e0-4f8c-b868-9627994bb7c3 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:14.310995 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-edc96931-33e0-4f8c-b868-9627994bb7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:14.386702 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1025/2053] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:14.387780 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc96931-33e0-4f8c-b868-9627994bb7c3 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:14.390229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-edc96931-33e0-4f8c-b868-9627994bb7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:14.428260 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-9bb79ffe-42d1-49c8-b76c-1a06d7dcaaf9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5775427a9b2e47fd82f56c6e7a76169b', 'request_id': 'req-9bb79ffe-42d1-49c8-b76c-1a06d7dcaaf9'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:14.437900 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-9bb79ffe-42d1-49c8-b76c-1a06d7dcaaf9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5775427a9b2e47fd82f56c6e7a76169b', 'request_id': 'req-9bb79ffe-42d1-49c8-b76c-1a06d7dcaaf9'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:14.488748 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc96931-33e0-4f8c-b868-9627994bb7c3 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:14.513752 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-9bb79ffe-42d1-49c8-b76c-1a06d7dcaaf9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5775427a9b2e47fd82f56c6e7a76169b', 'request_id': 'req-9bb79ffe-42d1-49c8-b76c-1a06d7dcaaf9'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:14.526941 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-9bb79ffe-42d1-49c8-b76c-1a06d7dcaaf9 admin admin] The token cache is being invalidate because project 5775427a9b2e47fd82f56c6e7a76169b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:47:14.527523 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-9bb79ffe-42d1-49c8-b76c-1a06d7dcaaf9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:14.528236 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-9bb79ffe-42d1-49c8-b76c-1a06d7dcaaf9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:14.529962 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1029/2054] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:47:14 2022] DELETE /identity/v3/projects/5775427a9b2e47fd82f56c6e7a76169b => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:14.531108 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb79ffe-42d1-49c8-b76c-1a06d7dcaaf9 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:14.532684 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-9bb79ffe-42d1-49c8-b76c-1a06d7dcaaf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:14.570999 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c5ad20-a553-4d09-839b-79fdd50f6494 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70f704a231a5474e8067a6c2dd88b355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:14.571739 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c5ad20-a553-4d09-839b-79fdd50f6494 service placement] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:14.572169 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c5ad20-a553-4d09-839b-79fdd50f6494 service placement] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:14.572475 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c5ad20-a553-4d09-839b-79fdd50f6494 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:14.574966 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-07c5ad20-a553-4d09-839b-79fdd50f6494 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:14.588263 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c5ad20-a553-4d09-839b-79fdd50f6494 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:14.592788 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c5ad20-a553-4d09-839b-79fdd50f6494 service placement] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:14.617872 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1026/2055] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:47:14.620435 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c5ad20-a553-4d09-839b-79fdd50f6494 service placement] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:14.622871 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-07c5ad20-a553-4d09-839b-79fdd50f6494 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:14.660726 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb79ffe-42d1-49c8-b76c-1a06d7dcaaf9 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:14.674717 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619bed08-25a3-4364-8d22-d0909fad9258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:14.675702 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619bed08-25a3-4364-8d22-d0909fad9258 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:14.676043 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619bed08-25a3-4364-8d22-d0909fad9258 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:14.676326 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619bed08-25a3-4364-8d22-d0909fad9258 admin admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:14.677034 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619bed08-25a3-4364-8d22-d0909fad9258 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:14.679003 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619bed08-25a3-4364-8d22-d0909fad9258 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:14.724092 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c5ad20-a553-4d09-839b-79fdd50f6494 service placement] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:14.728371 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c5a29-00f3-4a5c-8d27-9a84540e52ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:14.729098 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c5a29-00f3-4a5c-8d27-9a84540e52ad service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:14.729390 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c5a29-00f3-4a5c-8d27-9a84540e52ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:14.729732 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c5a29-00f3-4a5c-8d27-9a84540e52ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:14.731698 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d26c5a29-00f3-4a5c-8d27-9a84540e52ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:14.745148 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26c5a29-00f3-4a5c-8d27-9a84540e52ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:14.746669 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26c5a29-00f3-4a5c-8d27-9a84540e52ad service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:14.756902 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1027/2056] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:47:14.760833 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c07c0-0a73-4c87-b743-04ab239c1ac1 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:14.760833 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c07c0-0a73-4c87-b743-04ab239c1ac1 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:14.760833 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c07c0-0a73-4c87-b743-04ab239c1ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:14.784525 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-7c8c07c0-0a73-4c87-b743-04ab239c1ac1 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:14.785913 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-7c8c07c0-0a73-4c87-b743-04ab239c1ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:14.795593 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1028/2057] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:14.796696 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8c07c0-0a73-4c87-b743-04ab239c1ac1 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:14.798045 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-7c8c07c0-0a73-4c87-b743-04ab239c1ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:14.810336 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8c07c0-0a73-4c87-b743-04ab239c1ac1 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:14.816193 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56595f4-7575-4b36-b565-620af2d2b3e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:14.824322 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56595f4-7575-4b36-b565-620af2d2b3e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:14.824322 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56595f4-7575-4b36-b565-620af2d2b3e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:14.824322 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56595f4-7575-4b36-b565-620af2d2b3e7 admin admin] PATH_INFO: `/v3/projects/540cf87ca5364bf598ab16cdc3903a1c` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:14.824649 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1030/2058] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:47:14 2022] POST /identity/v3/users => generated 295 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:47:14.825121 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56595f4-7575-4b36-b565-620af2d2b3e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=540cf87ca5364bf598ab16cdc3903a1c)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:14.825121 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56595f4-7575-4b36-b565-620af2d2b3e7 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:14.825301 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf4abd-177e-4fd5-9b93-14da1e1aa76a None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:14.825301 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf4abd-177e-4fd5-9b93-14da1e1aa76a None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:14.825301 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf4abd-177e-4fd5-9b93-14da1e1aa76a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:14.857674 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-05bf4abd-177e-4fd5-9b93-14da1e1aa76a None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:14.858988 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-05bf4abd-177e-4fd5-9b93-14da1e1aa76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:14.867786 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1031/2059] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:14.872606 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf4abd-177e-4fd5-9b93-14da1e1aa76a None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:14.885204 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf4abd-177e-4fd5-9b93-14da1e1aa76a None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:14.891215 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0d0188-1b00-4af3-a398-be79354b3d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:14.892113 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d0188-1b00-4af3-a398-be79354b3d20 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:14.892665 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d0188-1b00-4af3-a398-be79354b3d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:14.893018 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d0188-1b00-4af3-a398-be79354b3d20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:14.893817 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0d0188-1b00-4af3-a398-be79354b3d20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:14.897621 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0d0188-1b00-4af3-a398-be79354b3d20 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:14.908553 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1032/2060] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:47:14.909404 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0d0188-1b00-4af3-a398-be79354b3d20 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:14.919523 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0d0188-1b00-4af3-a398-be79354b3d20 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:14.925459 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95985e2b-ff6c-411f-8f0c-de80c6dbe122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:14.929933 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95985e2b-ff6c-411f-8f0c-de80c6dbe122 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:14.929933 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95985e2b-ff6c-411f-8f0c-de80c6dbe122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:14.929933 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95985e2b-ff6c-411f-8f0c-de80c6dbe122 admin admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:14.929933 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95985e2b-ff6c-411f-8f0c-de80c6dbe122 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:47:14.929933 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95985e2b-ff6c-411f-8f0c-de80c6dbe122 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:14.929933 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95985e2b-ff6c-411f-8f0c-de80c6dbe122 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:14.945209 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1033/2061] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 16:47:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:47:14.946427 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95985e2b-ff6c-411f-8f0c-de80c6dbe122 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:14.962729 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95985e2b-ff6c-411f-8f0c-de80c6dbe122 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:14.985177 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-e56595f4-7575-4b36-b565-620af2d2b3e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '540cf87ca5364bf598ab16cdc3903a1c', 'request_id': 'req-e56595f4-7575-4b36-b565-620af2d2b3e7'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:15.000435 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-e56595f4-7575-4b36-b565-620af2d2b3e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '540cf87ca5364bf598ab16cdc3903a1c', 'request_id': 'req-e56595f4-7575-4b36-b565-620af2d2b3e7'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:15.025744 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-e56595f4-7575-4b36-b565-620af2d2b3e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '540cf87ca5364bf598ab16cdc3903a1c', 'request_id': 'req-e56595f4-7575-4b36-b565-620af2d2b3e7'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:15.036915 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70e4226-fd7b-4dfc-9674-994db74fbb32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:15.037942 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e4226-fd7b-4dfc-9674-994db74fbb32 admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:15.038230 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e4226-fd7b-4dfc-9674-994db74fbb32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:15.038507 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e4226-fd7b-4dfc-9674-994db74fbb32 admin admin] PATH_INFO: `/v3/projects/6492bf2a54994ad5a28c7de3eebfbbce/users/a82b452d649e46a58971bd35bc006cd0/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:15.059828 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70e4226-fd7b-4dfc-9674-994db74fbb32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6492bf2a54994ad5a28c7de3eebfbbce, user_id=a82b452d649e46a58971bd35bc006cd0, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:15.062794 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70e4226-fd7b-4dfc-9674-994db74fbb32 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:15.133721 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-e56595f4-7575-4b36-b565-620af2d2b3e7 admin admin] The token cache is being invalidate because project 540cf87ca5364bf598ab16cdc3903a1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:47:15.136206 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-e56595f4-7575-4b36-b565-620af2d2b3e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:15.136206 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-e56595f4-7575-4b36-b565-620af2d2b3e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:15.137007 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1029/2062] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:47:14 2022] DELETE /identity/v3/projects/540cf87ca5364bf598ab16cdc3903a1c => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:15.137789 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56595f4-7575-4b36-b565-620af2d2b3e7 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:15.139124 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e56595f4-7575-4b36-b565-620af2d2b3e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:15.153492 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56595f4-7575-4b36-b565-620af2d2b3e7 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:15.159159 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea984a79-853c-463f-998a-d833ea08ad57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:15.159873 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea984a79-853c-463f-998a-d833ea08ad57 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:15.160343 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea984a79-853c-463f-998a-d833ea08ad57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:15.160625 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea984a79-853c-463f-998a-d833ea08ad57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:15.161336 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea984a79-853c-463f-998a-d833ea08ad57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:15.163879 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea984a79-853c-463f-998a-d833ea08ad57 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:15.169170 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1034/2063] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:47:14 2022] PUT /identity/v3/projects/6492bf2a54994ad5a28c7de3eebfbbce/users/a82b452d649e46a58971bd35bc006cd0/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:15.171369 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ef53f-2c79-4654-9a9b-980517340e01 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:15.172497 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ef53f-2c79-4654-9a9b-980517340e01 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:15.173068 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ef53f-2c79-4654-9a9b-980517340e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:15.205804 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-445ef53f-2c79-4654-9a9b-980517340e01 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:15.207366 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-445ef53f-2c79-4654-9a9b-980517340e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:15.284670 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1035/2064] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:15.285747 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445ef53f-2c79-4654-9a9b-980517340e01 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:15.292599 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-445ef53f-2c79-4654-9a9b-980517340e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:15.298247 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1030/2065] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:47:15 2022] POST /identity/v3/projects => generated 375 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:47:15.300416 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436c1847-d5c7-4e18-9c96-497ebcdb9c0c None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:15.300896 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436c1847-d5c7-4e18-9c96-497ebcdb9c0c None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:15.301727 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436c1847-d5c7-4e18-9c96-497ebcdb9c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:15.313528 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445ef53f-2c79-4654-9a9b-980517340e01 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:15.330909 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-436c1847-d5c7-4e18-9c96-497ebcdb9c0c None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:15.332815 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-436c1847-d5c7-4e18-9c96-497ebcdb9c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:15.388694 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e13a082-3733-4d69-8bc5-74d4c49ce75b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:15.389378 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13a082-3733-4d69-8bc5-74d4c49ce75b admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:15.389668 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13a082-3733-4d69-8bc5-74d4c49ce75b admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:15.389938 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13a082-3733-4d69-8bc5-74d4c49ce75b admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:15.390924 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e13a082-3733-4d69-8bc5-74d4c49ce75b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:15.392846 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e13a082-3733-4d69-8bc5-74d4c49ce75b admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:15.405064 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1036/2066] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:15 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:47:15.411888 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1031/2067] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:15.412006 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e13a082-3733-4d69-8bc5-74d4c49ce75b admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:15.412101 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436c1847-d5c7-4e18-9c96-497ebcdb9c0c None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:15.412101 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-436c1847-d5c7-4e18-9c96-497ebcdb9c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:15.423775 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e13a082-3733-4d69-8bc5-74d4c49ce75b admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:15.426564 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436c1847-d5c7-4e18-9c96-497ebcdb9c0c None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:15.432552 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6889463a-3bca-478e-b961-fbca6f99a407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:15.432910 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889463a-3bca-478e-b961-fbca6f99a407 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:15.433215 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889463a-3bca-478e-b961-fbca6f99a407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:15.433384 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889463a-3bca-478e-b961-fbca6f99a407 admin admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:15.433801 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c11bba-fbc3-4d62-9139-c4e165111a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:15.434090 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6889463a-3bca-478e-b961-fbca6f99a407 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:15.434712 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c11bba-fbc3-4d62-9139-c4e165111a40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:15.435018 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c11bba-fbc3-4d62-9139-c4e165111a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:15.435291 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c11bba-fbc3-4d62-9139-c4e165111a40 admin admin] PATH_INFO: `/v3/projects/d9a814bcc8ed45c5bed3a0f5fa5531e0` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:15.436377 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6889463a-3bca-478e-b961-fbca6f99a407 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:15.440328 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c11bba-fbc3-4d62-9139-c4e165111a40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9a814bcc8ed45c5bed3a0f5fa5531e0)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:15.440328 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c11bba-fbc3-4d62-9139-c4e165111a40 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:15.546656 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1037/2068] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:47:15 2022] POST /identity/v3/users => generated 288 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:47:15.547394 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6889463a-3bca-478e-b961-fbca6f99a407 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:15.557977 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6889463a-3bca-478e-b961-fbca6f99a407 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:15.563062 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b3aca9-92bc-416f-a579-16f63da8512b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:15.564033 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b3aca9-92bc-416f-a579-16f63da8512b admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:15.564317 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b3aca9-92bc-416f-a579-16f63da8512b admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:15.564711 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b3aca9-92bc-416f-a579-16f63da8512b admin admin] PATH_INFO: `/v3/projects/6492bf2a54994ad5a28c7de3eebfbbce/users/a82b452d649e46a58971bd35bc006cd0/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:15.568583 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b3aca9-92bc-416f-a579-16f63da8512b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6492bf2a54994ad5a28c7de3eebfbbce, user_id=a82b452d649e46a58971bd35bc006cd0, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:15.577968 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b3aca9-92bc-416f-a579-16f63da8512b admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:15.584200 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1038/2069] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:47:15 2022] PUT /identity/v3/projects/6492bf2a54994ad5a28c7de3eebfbbce/users/a82b452d649e46a58971bd35bc006cd0/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:15.593315 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b3aca9-92bc-416f-a579-16f63da8512b admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:15.593445 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-c8c11bba-fbc3-4d62-9139-c4e165111a40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9a814bcc8ed45c5bed3a0f5fa5531e0', 'request_id': 'req-c8c11bba-fbc3-4d62-9139-c4e165111a40'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:15.593445 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-c8c11bba-fbc3-4d62-9139-c4e165111a40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9a814bcc8ed45c5bed3a0f5fa5531e0', 'request_id': 'req-c8c11bba-fbc3-4d62-9139-c4e165111a40'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:15.598532 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-c8c11bba-fbc3-4d62-9139-c4e165111a40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9a814bcc8ed45c5bed3a0f5fa5531e0', 'request_id': 'req-c8c11bba-fbc3-4d62-9139-c4e165111a40'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:15.652201 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b3aca9-92bc-416f-a579-16f63da8512b admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:15.657221 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6127abf6-0759-4fff-8bf5-f9c71c2d61b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:15.657934 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6127abf6-0759-4fff-8bf5-f9c71c2d61b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:15.658219 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6127abf6-0759-4fff-8bf5-f9c71c2d61b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:15.658498 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6127abf6-0759-4fff-8bf5-f9c71c2d61b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:15.659237 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6127abf6-0759-4fff-8bf5-f9c71c2d61b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:47:15.659575 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6127abf6-0759-4fff-8bf5-f9c71c2d61b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:15.661227 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6127abf6-0759-4fff-8bf5-f9c71c2d61b7 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:15.668998 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-c8c11bba-fbc3-4d62-9139-c4e165111a40 admin admin] The token cache is being invalidate because project d9a814bcc8ed45c5bed3a0f5fa5531e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:47:15.669895 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-c8c11bba-fbc3-4d62-9139-c4e165111a40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:15.670607 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-c8c11bba-fbc3-4d62-9139-c4e165111a40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:15.674417 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1032/2070] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:47:15 2022] DELETE /identity/v3/projects/d9a814bcc8ed45c5bed3a0f5fa5531e0 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:15.678154 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1039/2071] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 16:47:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:47:15.678753 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6127abf6-0759-4fff-8bf5-f9c71c2d61b7 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:15.681698 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d497ab62-9f4e-4e8b-b0b4-687a60bfcecf None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:15.682002 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d497ab62-9f4e-4e8b-b0b4-687a60bfcecf None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:15.682111 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-6127abf6-0759-4fff-8bf5-f9c71c2d61b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:15.682290 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d497ab62-9f4e-4e8b-b0b4-687a60bfcecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:15.701495 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6127abf6-0759-4fff-8bf5-f9c71c2d61b7 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:15.707246 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f898956-8e85-4a07-8a10-fa9bd0185ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:15.708132 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f898956-8e85-4a07-8a10-fa9bd0185ee2 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:15.708421 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f898956-8e85-4a07-8a10-fa9bd0185ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:15.708737 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f898956-8e85-4a07-8a10-fa9bd0185ee2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:15.709662 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f898956-8e85-4a07-8a10-fa9bd0185ee2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:15.711441 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f898956-8e85-4a07-8a10-fa9bd0185ee2 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:15.728887 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1040/2072] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:15 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:47:15.743711 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce0dd3-d44e-4521-8b89-d0291c6418cd None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:15.743711 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce0dd3-d44e-4521-8b89-d0291c6418cd None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:15.743711 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce0dd3-d44e-4521-8b89-d0291c6418cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:15.744459 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-d497ab62-9f4e-4e8b-b0b4-687a60bfcecf None None] MFA Rules not processed for user `a82b452d649e46a58971bd35bc006cd0`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:15.744459 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d497ab62-9f4e-4e8b-b0b4-687a60bfcecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:15.751735 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-28ce0dd3-d44e-4521-8b89-d0291c6418cd None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:15.753031 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-28ce0dd3-d44e-4521-8b89-d0291c6418cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:15.763846 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1041/2073] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:15.764681 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ce0dd3-d44e-4521-8b89-d0291c6418cd None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:15.766031 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-28ce0dd3-d44e-4521-8b89-d0291c6418cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:15.780059 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ce0dd3-d44e-4521-8b89-d0291c6418cd None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:15.787542 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af975e-ce71-4f7b-a36f-6626139b0316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:15.787542 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af975e-ce71-4f7b-a36f-6626139b0316 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:15.787542 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af975e-ce71-4f7b-a36f-6626139b0316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:15.787542 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af975e-ce71-4f7b-a36f-6626139b0316 admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:15.787952 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71af975e-ce71-4f7b-a36f-6626139b0316 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:15.789919 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71af975e-ce71-4f7b-a36f-6626139b0316 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:15.822700 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1033/2074] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:15 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:15.824181 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d497ab62-9f4e-4e8b-b0b4-687a60bfcecf None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:15.840932 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d497ab62-9f4e-4e8b-b0b4-687a60bfcecf None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:15.849082 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef73aa-2bbb-45a1-893d-b0d15c34493b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:15.851966 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef73aa-2bbb-45a1-893d-b0d15c34493b admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:15.851966 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef73aa-2bbb-45a1-893d-b0d15c34493b admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:15.851966 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef73aa-2bbb-45a1-893d-b0d15c34493b admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:15.851966 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ef73aa-2bbb-45a1-893d-b0d15c34493b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:15.853652 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ef73aa-2bbb-45a1-893d-b0d15c34493b admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:15.867114 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1034/2075] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:15 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:47:15.868536 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97861329-9b0b-4077-bfb2-b9ac3e7528bb None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:15.868977 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97861329-9b0b-4077-bfb2-b9ac3e7528bb None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:15.869511 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97861329-9b0b-4077-bfb2-b9ac3e7528bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:15.880815 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1042/2076] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:47:15 2022] POST /identity/v3/projects => generated 371 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:47:15.882400 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af975e-ce71-4f7b-a36f-6626139b0316 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:15.884453 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-71af975e-ce71-4f7b-a36f-6626139b0316 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:15.890168 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-97861329-9b0b-4077-bfb2-b9ac3e7528bb None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:15.891385 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-97861329-9b0b-4077-bfb2-b9ac3e7528bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:15.952067 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af975e-ce71-4f7b-a36f-6626139b0316 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:15.955789 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc8dfd-d4da-438b-99fb-b43efba2673f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:15.956503 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc8dfd-d4da-438b-99fb-b43efba2673f service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:15.956918 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc8dfd-d4da-438b-99fb-b43efba2673f service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:15.957205 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc8dfd-d4da-438b-99fb-b43efba2673f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:15.959055 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-abcc8dfd-d4da-438b-99fb-b43efba2673f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:15.962439 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1035/2077] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:15.963810 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97861329-9b0b-4077-bfb2-b9ac3e7528bb None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:15.974395 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97861329-9b0b-4077-bfb2-b9ac3e7528bb None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:15.980255 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb43ea42-1f58-4039-81c7-2d8072b6d2d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:15.981207 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43ea42-1f58-4039-81c7-2d8072b6d2d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:15.981489 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43ea42-1f58-4039-81c7-2d8072b6d2d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:15.981770 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43ea42-1f58-4039-81c7-2d8072b6d2d0 admin admin] PATH_INFO: `/v3/projects/ceef800c47564c23a140b7f0e2382a02/users/f6126e9b566843beb1bc1e813a5f339f/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:16.000285 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb43ea42-1f58-4039-81c7-2d8072b6d2d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ceef800c47564c23a140b7f0e2382a02, user_id=f6126e9b566843beb1bc1e813a5f339f, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:16.003007 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb43ea42-1f58-4039-81c7-2d8072b6d2d0 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:16.026701 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcc8dfd-d4da-438b-99fb-b43efba2673f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:16.028291 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcc8dfd-d4da-438b-99fb-b43efba2673f service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:16.038226 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1043/2078] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:47:16.039010 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc8dfd-d4da-438b-99fb-b43efba2673f service cinder] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:16.050694 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc8dfd-d4da-438b-99fb-b43efba2673f service cinder] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:16.055661 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99699fbe-08af-4969-a3d7-2dd961e9add4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:16.056492 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99699fbe-08af-4969-a3d7-2dd961e9add4 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:16.056784 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99699fbe-08af-4969-a3d7-2dd961e9add4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:16.057057 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99699fbe-08af-4969-a3d7-2dd961e9add4 admin admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:16.057892 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99699fbe-08af-4969-a3d7-2dd961e9add4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:16.059854 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99699fbe-08af-4969-a3d7-2dd961e9add4 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:16.081154 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1036/2079] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:47:15 2022] PUT /identity/v3/projects/ceef800c47564c23a140b7f0e2382a02/users/f6126e9b566843beb1bc1e813a5f339f/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:16.088882 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb43ea42-1f58-4039-81c7-2d8072b6d2d0 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:16.159617 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb43ea42-1f58-4039-81c7-2d8072b6d2d0 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:16.169098 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75ec20e-01a8-49f2-9e8f-28dcd18d80a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:16.169804 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ec20e-01a8-49f2-9e8f-28dcd18d80a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:16.170086 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ec20e-01a8-49f2-9e8f-28dcd18d80a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:16.170358 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ec20e-01a8-49f2-9e8f-28dcd18d80a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:16.171139 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75ec20e-01a8-49f2-9e8f-28dcd18d80a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:47:16.171486 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75ec20e-01a8-49f2-9e8f-28dcd18d80a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:16.173504 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75ec20e-01a8-49f2-9e8f-28dcd18d80a6 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:16.186552 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1037/2080] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 16:47:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:47:16.190429 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8379824-1a6e-49ac-9b69-ba61b06327f5 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:16.190429 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8379824-1a6e-49ac-9b69-ba61b06327f5 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:16.190429 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8379824-1a6e-49ac-9b69-ba61b06327f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:16.217188 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-d8379824-1a6e-49ac-9b69-ba61b06327f5 None None] MFA Rules not processed for user `f6126e9b566843beb1bc1e813a5f339f`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:16.218408 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d8379824-1a6e-49ac-9b69-ba61b06327f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:16.245349 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1044/2081] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:47:16 2022] POST /identity/v3/users => generated 286 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:47:16.246221 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99699fbe-08af-4969-a3d7-2dd961e9add4 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:16.256678 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99699fbe-08af-4969-a3d7-2dd961e9add4 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:16.267570 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54435161-ff67-4aa7-8213-8493cd315c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:16.268287 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54435161-ff67-4aa7-8213-8493cd315c21 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:16.268656 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54435161-ff67-4aa7-8213-8493cd315c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:16.268940 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54435161-ff67-4aa7-8213-8493cd315c21 admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:16.269645 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54435161-ff67-4aa7-8213-8493cd315c21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:16.271789 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54435161-ff67-4aa7-8213-8493cd315c21 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:16.310653 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1038/2082] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:16.311635 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8379824-1a6e-49ac-9b69-ba61b06327f5 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:16.330818 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8379824-1a6e-49ac-9b69-ba61b06327f5 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:16.335602 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4a4d89-0faa-479b-b3fd-a341dd09dd0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:16.336281 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a4d89-0faa-479b-b3fd-a341dd09dd0a admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:16.336725 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a4d89-0faa-479b-b3fd-a341dd09dd0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:16.337107 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a4d89-0faa-479b-b3fd-a341dd09dd0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:16.338320 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4a4d89-0faa-479b-b3fd-a341dd09dd0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:16.339981 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4a4d89-0faa-479b-b3fd-a341dd09dd0a admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:16.351218 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1039/2083] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:16 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:47:16.352254 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4a4d89-0faa-479b-b3fd-a341dd09dd0a admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:16.354784 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-ca4a4d89-0faa-479b-b3fd-a341dd09dd0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:16.424484 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1045/2084] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:47:16 2022] POST /identity/v3/projects => generated 383 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:47:16.426969 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54435161-ff67-4aa7-8213-8493cd315c21 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:16.429848 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:16.468421 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4a4d89-0faa-479b-b3fd-a341dd09dd0a admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:16.531320 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54435161-ff67-4aa7-8213-8493cd315c21 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:16.536585 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e90e58-7857-42ee-aa5b-fb0ac7650b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:16.540933 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e90e58-7857-42ee-aa5b-fb0ac7650b99 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:16.540933 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e90e58-7857-42ee-aa5b-fb0ac7650b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:16.540933 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e90e58-7857-42ee-aa5b-fb0ac7650b99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:16.541305 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e46e0e1-02df-4d77-8b81-e749c17162c8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0887908bcb0d4072bf6acb4cbaf1af83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:16.541412 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e90e58-7857-42ee-aa5b-fb0ac7650b99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:16.541514 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e46e0e1-02df-4d77-8b81-e749c17162c8 service glance] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:16.541514 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e46e0e1-02df-4d77-8b81-e749c17162c8 service glance] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:16.541514 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e46e0e1-02df-4d77-8b81-e749c17162c8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:16.541830 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e90e58-7857-42ee-aa5b-fb0ac7650b99 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:16.552049 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1046/2085] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:16 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:47:16.552435 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e90e58-7857-42ee-aa5b-fb0ac7650b99 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:16.565621 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e90e58-7857-42ee-aa5b-fb0ac7650b99 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:16.571175 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de1922-4c08-411c-b29e-88de92b8aebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:16.572174 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de1922-4c08-411c-b29e-88de92b8aebc admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:16.572460 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de1922-4c08-411c-b29e-88de92b8aebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:16.572740 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de1922-4c08-411c-b29e-88de92b8aebc admin admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:16.573724 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8de1922-4c08-411c-b29e-88de92b8aebc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:16.575928 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8de1922-4c08-411c-b29e-88de92b8aebc admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:16.609845 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e46e0e1-02df-4d77-8b81-e749c17162c8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:16.611655 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e46e0e1-02df-4d77-8b81-e749c17162c8 service glance] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:16.622232 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1040/2086] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:16 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:47:16.630470 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e46e0e1-02df-4d77-8b81-e749c17162c8 service glance] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:16.635937 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e46e0e1-02df-4d77-8b81-e749c17162c8 service glance] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:16.640816 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15862acd-78cb-4987-9f68-f819e55442d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:16.641841 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15862acd-78cb-4987-9f68-f819e55442d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:16.642135 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15862acd-78cb-4987-9f68-f819e55442d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:16.642478 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15862acd-78cb-4987-9f68-f819e55442d6 admin admin] PATH_INFO: `/v3/projects/d0e7e5bdb6c44cdbaa611ef85f78209c/users/0f361fe217a947b99163a8c3876113ce/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:16.672946 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15862acd-78cb-4987-9f68-f819e55442d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0e7e5bdb6c44cdbaa611ef85f78209c, user_id=0f361fe217a947b99163a8c3876113ce, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:16.672946 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15862acd-78cb-4987-9f68-f819e55442d6 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:16.701298 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:16.705051 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1047/2087] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:47:16 2022] POST /identity/v3/users => generated 292 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:47:16.709509 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de1922-4c08-411c-b29e-88de92b8aebc admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:16.721681 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de1922-4c08-411c-b29e-88de92b8aebc admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:16.726794 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4715487-ed5b-4508-9bfb-26f50b52ff39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:16.729698 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4715487-ed5b-4508-9bfb-26f50b52ff39 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:16.729698 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4715487-ed5b-4508-9bfb-26f50b52ff39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:16.729698 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4715487-ed5b-4508-9bfb-26f50b52ff39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:16.729698 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4715487-ed5b-4508-9bfb-26f50b52ff39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:16.731144 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4715487-ed5b-4508-9bfb-26f50b52ff39 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:16.747080 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1048/2088] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:16 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:47:16.748260 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4715487-ed5b-4508-9bfb-26f50b52ff39 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:16.755780 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1041/2089] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:47:16 2022] PUT /identity/v3/projects/d0e7e5bdb6c44cdbaa611ef85f78209c/users/0f361fe217a947b99163a8c3876113ce/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:16.762600 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4715487-ed5b-4508-9bfb-26f50b52ff39 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:16.772319 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37727b8-0674-4caf-843f-46adf6b54161 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:16.772909 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37727b8-0674-4caf-843f-46adf6b54161 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:16.773196 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37727b8-0674-4caf-843f-46adf6b54161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:16.843946 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-e37727b8-0674-4caf-843f-46adf6b54161 None None] MFA Rules not processed for user `0f361fe217a947b99163a8c3876113ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:16.844586 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e37727b8-0674-4caf-843f-46adf6b54161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:16.855246 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a140588a-3dce-4838-a3a7-e059fdb17bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:16.857634 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a140588a-3dce-4838-a3a7-e059fdb17bf3 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:16.860743 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a140588a-3dce-4838-a3a7-e059fdb17bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:16.860743 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a140588a-3dce-4838-a3a7-e059fdb17bf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:16.860743 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a140588a-3dce-4838-a3a7-e059fdb17bf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:16.862933 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a140588a-3dce-4838-a3a7-e059fdb17bf3 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:16.879450 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1049/2090] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:16 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:47:16.884527 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a140588a-3dce-4838-a3a7-e059fdb17bf3 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:16.901937 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a140588a-3dce-4838-a3a7-e059fdb17bf3 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:16.910461 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a929af-ea7e-40c7-9b1b-3518efc5e01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:16.912084 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a929af-ea7e-40c7-9b1b-3518efc5e01a admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:16.912377 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a929af-ea7e-40c7-9b1b-3518efc5e01a admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:16.912659 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a929af-ea7e-40c7-9b1b-3518efc5e01a admin admin] PATH_INFO: `/v3/projects/8482fe90b80d4758b0dec096cb492869/users/7fb5d5bca45845de8c487e3dca445e22/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:16.933368 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a929af-ea7e-40c7-9b1b-3518efc5e01a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8482fe90b80d4758b0dec096cb492869, user_id=7fb5d5bca45845de8c487e3dca445e22, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:16.934472 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a929af-ea7e-40c7-9b1b-3518efc5e01a admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:16.951537 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1042/2091] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:47:16.954340 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37727b8-0674-4caf-843f-46adf6b54161 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:16.974824 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37727b8-0674-4caf-843f-46adf6b54161 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:16.982219 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6943c7f-0449-4453-a055-3f4422a0ae07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:16.982958 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6943c7f-0449-4453-a055-3f4422a0ae07 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:16.983245 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6943c7f-0449-4453-a055-3f4422a0ae07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:16.983546 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6943c7f-0449-4453-a055-3f4422a0ae07 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:16.984572 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6943c7f-0449-4453-a055-3f4422a0ae07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:16.988067 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6943c7f-0449-4453-a055-3f4422a0ae07 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:17.018729 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1050/2092] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:47:16 2022] PUT /identity/v3/projects/8482fe90b80d4758b0dec096cb492869/users/7fb5d5bca45845de8c487e3dca445e22/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:17.033427 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f394b9-70ad-400d-9ef7-1957ade74ee9 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:17.033724 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f394b9-70ad-400d-9ef7-1957ade74ee9 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:17.034134 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f394b9-70ad-400d-9ef7-1957ade74ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:17.062136 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-d5f394b9-70ad-400d-9ef7-1957ade74ee9 None None] MFA Rules not processed for user `7fb5d5bca45845de8c487e3dca445e22`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:17.063407 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-d5f394b9-70ad-400d-9ef7-1957ade74ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:17.105131 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1043/2093] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:47:16 2022] POST /identity/v3/projects => generated 371 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:47:17.109652 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6943c7f-0449-4453-a055-3f4422a0ae07 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:17.133783 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1051/2094] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:17.148627 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f394b9-70ad-400d-9ef7-1957ade74ee9 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:17.184435 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6943c7f-0449-4453-a055-3f4422a0ae07 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:17.189331 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18a1506-bfed-43ba-bb55-e59db1e1c55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:17.190075 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a1506-bfed-43ba-bb55-e59db1e1c55a admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:17.190355 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a1506-bfed-43ba-bb55-e59db1e1c55a admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:17.190657 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a1506-bfed-43ba-bb55-e59db1e1c55a admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:17.191341 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18a1506-bfed-43ba-bb55-e59db1e1c55a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:17.193385 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18a1506-bfed-43ba-bb55-e59db1e1c55a admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:17.211156 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f394b9-70ad-400d-9ef7-1957ade74ee9 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:17.263363 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac3e9a3-636d-41b5-9144-5fd56c23a0dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:17.264077 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3e9a3-636d-41b5-9144-5fd56c23a0dd service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:17.264361 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3e9a3-636d-41b5-9144-5fd56c23a0dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:17.264637 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3e9a3-636d-41b5-9144-5fd56c23a0dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:17.311843 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1044/2095] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:47:17 2022] POST /identity/v3/users => generated 292 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:47:17.312968 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18a1506-bfed-43ba-bb55-e59db1e1c55a admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:17.323963 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18a1506-bfed-43ba-bb55-e59db1e1c55a admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:17.329454 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f758ce0-971f-4d8e-ad4d-1d99e08dcc59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:17.329810 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac3e9a3-636d-41b5-9144-5fd56c23a0dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:17.330159 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f758ce0-971f-4d8e-ad4d-1d99e08dcc59 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:17.330445 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f758ce0-971f-4d8e-ad4d-1d99e08dcc59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:17.330723 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f758ce0-971f-4d8e-ad4d-1d99e08dcc59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:17.331267 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac3e9a3-636d-41b5-9144-5fd56c23a0dd service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:17.331640 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f758ce0-971f-4d8e-ad4d-1d99e08dcc59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:17.333350 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f758ce0-971f-4d8e-ad4d-1d99e08dcc59 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:17.341379 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1052/2096] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:47:17.346835 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1045/2097] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:47:17.349916 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac3e9a3-636d-41b5-9144-5fd56c23a0dd service cinder] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:17.361406 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac3e9a3-636d-41b5-9144-5fd56c23a0dd service cinder] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:17.366056 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e85f1bd-7a65-47be-8155-46c03ad50096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:17.367013 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e85f1bd-7a65-47be-8155-46c03ad50096 admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:17.367297 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e85f1bd-7a65-47be-8155-46c03ad50096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:17.367659 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e85f1bd-7a65-47be-8155-46c03ad50096 admin admin] PATH_INFO: `/v3/projects/f00284e2088c48b492bb6eb49cfcac49/users/948d7c5da7254ed2840787c8a199105a/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:17.384491 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e85f1bd-7a65-47be-8155-46c03ad50096 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f00284e2088c48b492bb6eb49cfcac49, user_id=948d7c5da7254ed2840787c8a199105a, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:17.387321 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e85f1bd-7a65-47be-8155-46c03ad50096 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:17.469578 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1053/2098] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:47:17 2022] PUT /identity/v3/projects/f00284e2088c48b492bb6eb49cfcac49/users/948d7c5da7254ed2840787c8a199105a/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:47:17.473662 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f758ce0-971f-4d8e-ad4d-1d99e08dcc59 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:17.486877 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f758ce0-971f-4d8e-ad4d-1d99e08dcc59 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:17.570706 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e06d2b-a341-4dcf-af1f-08c86988b97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:17.571912 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e06d2b-a341-4dcf-af1f-08c86988b97c admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:17.572621 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e06d2b-a341-4dcf-af1f-08c86988b97c admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:17.572935 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e06d2b-a341-4dcf-af1f-08c86988b97c admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:17.573927 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e06d2b-a341-4dcf-af1f-08c86988b97c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:17.583433 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e06d2b-a341-4dcf-af1f-08c86988b97c admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:17.599627 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1046/2099] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:17 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:47:17.600909 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e85f1bd-7a65-47be-8155-46c03ad50096 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:17.614534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e85f1bd-7a65-47be-8155-46c03ad50096 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:17.619934 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a765d61d-abeb-4c49-9f29-0f2a9496463f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:17.621115 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765d61d-abeb-4c49-9f29-0f2a9496463f admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:17.621514 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765d61d-abeb-4c49-9f29-0f2a9496463f admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:17.621751 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765d61d-abeb-4c49-9f29-0f2a9496463f admin admin] PATH_INFO: `/v3/projects/f00284e2088c48b492bb6eb49cfcac49/users/948d7c5da7254ed2840787c8a199105a/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:17.624034 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a765d61d-abeb-4c49-9f29-0f2a9496463f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f00284e2088c48b492bb6eb49cfcac49, user_id=948d7c5da7254ed2840787c8a199105a, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:17.626887 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a765d61d-abeb-4c49-9f29-0f2a9496463f admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:17.645899 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1054/2100] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:47:17 2022] PUT /identity/v3/projects/f00284e2088c48b492bb6eb49cfcac49/users/948d7c5da7254ed2840787c8a199105a/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:17.658387 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aa6e80-6de3-4906-bff5-89202e39b5af None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:17.658875 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aa6e80-6de3-4906-bff5-89202e39b5af None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:17.659245 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aa6e80-6de3-4906-bff5-89202e39b5af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:17.690115 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-e1aa6e80-6de3-4906-bff5-89202e39b5af None None] MFA Rules not processed for user `948d7c5da7254ed2840787c8a199105a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:17.691408 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e1aa6e80-6de3-4906-bff5-89202e39b5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:17.805117 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1047/2101] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:17 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:17.813275 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a765d61d-abeb-4c49-9f29-0f2a9496463f admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:17.815245 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-a765d61d-abeb-4c49-9f29-0f2a9496463f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:17.886759 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a765d61d-abeb-4c49-9f29-0f2a9496463f admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:17.890800 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d531bf3b-99b7-4a48-9c94-71dcc8c8034d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:17.891698 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d531bf3b-99b7-4a48-9c94-71dcc8c8034d service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:17.892050 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d531bf3b-99b7-4a48-9c94-71dcc8c8034d service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:17.892399 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d531bf3b-99b7-4a48-9c94-71dcc8c8034d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:17.905203 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d531bf3b-99b7-4a48-9c94-71dcc8c8034d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:17.907127 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d531bf3b-99b7-4a48-9c94-71dcc8c8034d service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:17.918651 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1055/2102] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:17 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:47:18.036097 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aa6e80-6de3-4906-bff5-89202e39b5af None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:18.037557 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e1aa6e80-6de3-4906-bff5-89202e39b5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:18.141872 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aa6e80-6de3-4906-bff5-89202e39b5af None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:18.147111 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7637a5-fb50-49ac-b8fa-c0aa05aa32f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:18.147954 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7637a5-fb50-49ac-b8fa-c0aa05aa32f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:18.148204 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7637a5-fb50-49ac-b8fa-c0aa05aa32f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:18.148478 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7637a5-fb50-49ac-b8fa-c0aa05aa32f7 admin admin] PATH_INFO: `/v3/users/39607830620d46c08f574ee6bdbc5575` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:18.150383 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7637a5-fb50-49ac-b8fa-c0aa05aa32f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39607830620d46c08f574ee6bdbc5575)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:18.152603 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7637a5-fb50-49ac-b8fa-c0aa05aa32f7 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:18.632754 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-bc7637a5-fb50-49ac-b8fa-c0aa05aa32f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39607830620d46c08f574ee6bdbc5575'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:18.725737 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:18.760049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-bc7637a5-fb50-49ac-b8fa-c0aa05aa32f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39607830620d46c08f574ee6bdbc5575'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:18.769572 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-bc7637a5-fb50-49ac-b8fa-c0aa05aa32f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39607830620d46c08f574ee6bdbc5575'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:18.789089 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-bc7637a5-fb50-49ac-b8fa-c0aa05aa32f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39607830620d46c08f574ee6bdbc5575'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:18.790016 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-bc7637a5-fb50-49ac-b8fa-c0aa05aa32f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39607830620d46c08f574ee6bdbc5575'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:18.798300 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1048/2103] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:47:18 2022] DELETE /identity/v3/users/39607830620d46c08f574ee6bdbc5575 => generated 0 bytes in 763 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:18.802682 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d1725c-13a0-40ad-a3b6-c18642e44393 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:18.802973 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d1725c-13a0-40ad-a3b6-c18642e44393 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:18.803348 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d1725c-13a0-40ad-a3b6-c18642e44393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:18.821545 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-c1d1725c-13a0-40ad-a3b6-c18642e44393 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:18.825345 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-c1d1725c-13a0-40ad-a3b6-c18642e44393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:18.898478 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1056/2104] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:18.900182 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:18.910088 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7637a5-fb50-49ac-b8fa-c0aa05aa32f7 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:18.911807 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-bc7637a5-fb50-49ac-b8fa-c0aa05aa32f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:18.991262 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7637a5-fb50-49ac-b8fa-c0aa05aa32f7 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:18.995237 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b224da-81c5-4fe2-8e91-d886e07dc431 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:18.998317 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b224da-81c5-4fe2-8e91-d886e07dc431 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:18.998317 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b224da-81c5-4fe2-8e91-d886e07dc431 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:18.998317 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b224da-81c5-4fe2-8e91-d886e07dc431 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:19.007269 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b224da-81c5-4fe2-8e91-d886e07dc431 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:19.009785 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b224da-81c5-4fe2-8e91-d886e07dc431 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:19.021292 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1049/2105] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:47:19.483863 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d1725c-13a0-40ad-a3b6-c18642e44393 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:19.483863 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-c1d1725c-13a0-40ad-a3b6-c18642e44393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:19.496813 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d1725c-13a0-40ad-a3b6-c18642e44393 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:19.506302 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1a9ea9-0539-49d6-901b-0530cad015b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:19.506302 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a9ea9-0539-49d6-901b-0530cad015b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:19.506302 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a9ea9-0539-49d6-901b-0530cad015b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:19.506302 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a9ea9-0539-49d6-901b-0530cad015b1 admin admin] PATH_INFO: `/v3/users/0f361fe217a947b99163a8c3876113ce` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:19.508393 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1a9ea9-0539-49d6-901b-0530cad015b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f361fe217a947b99163a8c3876113ce)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:19.510639 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1a9ea9-0539-49d6-901b-0530cad015b1 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:19.671672 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b224da-81c5-4fe2-8e91-d886e07dc431 service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:19.673463 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-66b224da-81c5-4fe2-8e91-d886e07dc431 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:19.690194 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b224da-81c5-4fe2-8e91-d886e07dc431 service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:19.695359 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a108e0-fe46-4458-84e0-242d4ca2b269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:19.696413 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a108e0-fe46-4458-84e0-242d4ca2b269 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:19.696849 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a108e0-fe46-4458-84e0-242d4ca2b269 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:19.699120 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a108e0-fe46-4458-84e0-242d4ca2b269 admin admin] PATH_INFO: `/v3/projects/bc192e7e5a3b4a8da04f8f0b53105592` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:19.701108 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a108e0-fe46-4458-84e0-242d4ca2b269 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc192e7e5a3b4a8da04f8f0b53105592)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:19.706881 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a108e0-fe46-4458-84e0-242d4ca2b269 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:19.780729 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-7f1a9ea9-0539-49d6-901b-0530cad015b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f361fe217a947b99163a8c3876113ce'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:19.900187 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-c5a108e0-fe46-4458-84e0-242d4ca2b269 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc192e7e5a3b4a8da04f8f0b53105592', 'request_id': 'req-c5a108e0-fe46-4458-84e0-242d4ca2b269'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:19.906763 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-c5a108e0-fe46-4458-84e0-242d4ca2b269 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc192e7e5a3b4a8da04f8f0b53105592', 'request_id': 'req-c5a108e0-fe46-4458-84e0-242d4ca2b269'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:19.913536 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-c5a108e0-fe46-4458-84e0-242d4ca2b269 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc192e7e5a3b4a8da04f8f0b53105592', 'request_id': 'req-c5a108e0-fe46-4458-84e0-242d4ca2b269'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:19.920975 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-7f1a9ea9-0539-49d6-901b-0530cad015b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f361fe217a947b99163a8c3876113ce'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:19.923405 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-7f1a9ea9-0539-49d6-901b-0530cad015b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f361fe217a947b99163a8c3876113ce'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:19.924070 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-7f1a9ea9-0539-49d6-901b-0530cad015b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f361fe217a947b99163a8c3876113ce'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:19.933676 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-7f1a9ea9-0539-49d6-901b-0530cad015b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f361fe217a947b99163a8c3876113ce'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:19.957177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1057/2106] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:47:19 2022] DELETE /identity/v3/users/0f361fe217a947b99163a8c3876113ce => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:19.963033 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a6604b-0a57-44c8-adc9-f59a5507571f None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:19.965328 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a6604b-0a57-44c8-adc9-f59a5507571f None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:19.965328 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a6604b-0a57-44c8-adc9-f59a5507571f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:19.983878 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-e4a6604b-0a57-44c8-adc9-f59a5507571f None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:19.985152 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-e4a6604b-0a57-44c8-adc9-f59a5507571f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:20.046037 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1058/2107] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:47:20.055526 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a6604b-0a57-44c8-adc9-f59a5507571f None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:20.057510 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-e4a6604b-0a57-44c8-adc9-f59a5507571f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:20.077075 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-c5a108e0-fe46-4458-84e0-242d4ca2b269 admin admin] The token cache is being invalidate because project bc192e7e5a3b4a8da04f8f0b53105592 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:47:20.078163 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-c5a108e0-fe46-4458-84e0-242d4ca2b269 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:20.082453 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-c5a108e0-fe46-4458-84e0-242d4ca2b269 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:20.092676 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1050/2108] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:47:19 2022] DELETE /identity/v3/projects/bc192e7e5a3b4a8da04f8f0b53105592 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:20.097354 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa680a3-1812-43ee-ba49-6d8c35419269 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:20.097768 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa680a3-1812-43ee-ba49-6d8c35419269 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:20.098148 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa680a3-1812-43ee-ba49-6d8c35419269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:20.128324 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a6604b-0a57-44c8-adc9-f59a5507571f None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:20.133645 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c2235a-8c51-4eaa-958d-929a61f39c87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:20.133645 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2235a-8c51-4eaa-958d-929a61f39c87 service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:20.133645 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2235a-8c51-4eaa-958d-929a61f39c87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:20.133645 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2235a-8c51-4eaa-958d-929a61f39c87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:20.138555 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-7aa680a3-1812-43ee-ba49-6d8c35419269 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:20.139670 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-a6c2235a-8c51-4eaa-958d-929a61f39c87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:20.139852 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-7aa680a3-1812-43ee-ba49-6d8c35419269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:20.160930 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1051/2109] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:20.170282 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6083db-e873-4b4d-825e-330b6aac11f7 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:20.170578 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6083db-e873-4b4d-825e-330b6aac11f7 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:20.170957 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6083db-e873-4b4d-825e-330b6aac11f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:20.176424 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:20.180220 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c2235a-8c51-4eaa-958d-929a61f39c87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:20.183282 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c2235a-8c51-4eaa-958d-929a61f39c87 service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:20.203424 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1059/2110] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:47:20.204343 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-6b6083db-e873-4b4d-825e-330b6aac11f7 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:20.205595 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-6b6083db-e873-4b4d-825e-330b6aac11f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:20.215839 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1052/2111] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:20.220144 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c2235a-8c51-4eaa-958d-929a61f39c87 service neutron] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:20.232309 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c2235a-8c51-4eaa-958d-929a61f39c87 service neutron] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:20.237153 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02885d3-49cd-472e-a56d-c13183b12996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:20.237880 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02885d3-49cd-472e-a56d-c13183b12996 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:20.238283 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02885d3-49cd-472e-a56d-c13183b12996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:20.238451 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02885d3-49cd-472e-a56d-c13183b12996 admin admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:20.239161 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02885d3-49cd-472e-a56d-c13183b12996 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:47:20.239479 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02885d3-49cd-472e-a56d-c13183b12996 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:20.241369 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02885d3-49cd-472e-a56d-c13183b12996 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:20.255132 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1060/2112] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 16:47:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:47:20.261299 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6083db-e873-4b4d-825e-330b6aac11f7 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:20.271476 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6083db-e873-4b4d-825e-330b6aac11f7 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:20.278369 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af820fc-8ff1-4302-9604-9ed56e6e642b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:20.279052 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af820fc-8ff1-4302-9604-9ed56e6e642b admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:20.279331 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af820fc-8ff1-4302-9604-9ed56e6e642b admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:20.279974 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af820fc-8ff1-4302-9604-9ed56e6e642b admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:20.281631 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af820fc-8ff1-4302-9604-9ed56e6e642b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:20.283598 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af820fc-8ff1-4302-9604-9ed56e6e642b admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:20.414881 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1053/2113] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:47:20 2022] POST /identity/v3/projects => generated 371 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:47:20.416901 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02885d3-49cd-472e-a56d-c13183b12996 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:20.428328 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02885d3-49cd-472e-a56d-c13183b12996 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:20.497205 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d31323-fdee-4b16-94cb-e8cbf0784d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:20.498001 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d31323-fdee-4b16-94cb-e8cbf0784d95 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:20.498355 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d31323-fdee-4b16-94cb-e8cbf0784d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:20.498706 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d31323-fdee-4b16-94cb-e8cbf0784d95 admin admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:20.499513 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d31323-fdee-4b16-94cb-e8cbf0784d95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:20.501591 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d31323-fdee-4b16-94cb-e8cbf0784d95 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:20.597816 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1061/2114] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:47:20 2022] POST /identity/v3/users => generated 286 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:47:20.602170 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af820fc-8ff1-4302-9604-9ed56e6e642b admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:20.613246 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af820fc-8ff1-4302-9604-9ed56e6e642b admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:20.618106 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cc2f0b-1c16-4866-92da-0d8705392bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:20.618847 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc2f0b-1c16-4866-92da-0d8705392bd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:20.619136 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc2f0b-1c16-4866-92da-0d8705392bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:20.619415 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc2f0b-1c16-4866-92da-0d8705392bd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:20.620186 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cc2f0b-1c16-4866-92da-0d8705392bd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:20.621793 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cc2f0b-1c16-4866-92da-0d8705392bd9 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:20.633396 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1054/2115] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:47:20.638242 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d31323-fdee-4b16-94cb-e8cbf0784d95 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:20.648673 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d31323-fdee-4b16-94cb-e8cbf0784d95 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:20.655146 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6153d0-ade7-4830-91bb-fcf59d6b53f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:20.655872 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6153d0-ade7-4830-91bb-fcf59d6b53f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:20.656165 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6153d0-ade7-4830-91bb-fcf59d6b53f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:20.656479 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6153d0-ade7-4830-91bb-fcf59d6b53f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:20.657233 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6153d0-ade7-4830-91bb-fcf59d6b53f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:20.658886 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6153d0-ade7-4830-91bb-fcf59d6b53f7 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:20.670161 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1062/2116] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:47:20.674739 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cc2f0b-1c16-4866-92da-0d8705392bd9 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:20.686153 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cc2f0b-1c16-4866-92da-0d8705392bd9 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:20.691845 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6153d0-ade7-4830-91bb-fcf59d6b53f7 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:20.691982 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e72f549-4d1a-498a-8041-dffa9693994e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:20.693106 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e72f549-4d1a-498a-8041-dffa9693994e admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:20.693561 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e72f549-4d1a-498a-8041-dffa9693994e admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:20.694022 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e72f549-4d1a-498a-8041-dffa9693994e admin admin] PATH_INFO: `/v3/projects/fcf58f51026c4ee2bb9d8d9452208dce/users/04dea46a9cbc49afa81bc333f34c50db/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:20.694918 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-5c6153d0-ade7-4830-91bb-fcf59d6b53f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:20.712896 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e72f549-4d1a-498a-8041-dffa9693994e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcf58f51026c4ee2bb9d8d9452208dce, user_id=04dea46a9cbc49afa81bc333f34c50db, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:20.714806 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6153d0-ade7-4830-91bb-fcf59d6b53f7 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:20.715810 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e72f549-4d1a-498a-8041-dffa9693994e admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:20.718497 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10336ae-4c67-444e-ad62-e9a49bcd2fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:20.720065 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10336ae-4c67-444e-ad62-e9a49bcd2fd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:20.720421 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10336ae-4c67-444e-ad62-e9a49bcd2fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:20.720765 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10336ae-4c67-444e-ad62-e9a49bcd2fd1 admin admin] PATH_INFO: `/v3/projects/d0e7e5bdb6c44cdbaa611ef85f78209c` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:20.722183 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10336ae-4c67-444e-ad62-e9a49bcd2fd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0e7e5bdb6c44cdbaa611ef85f78209c)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:20.724411 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10336ae-4c67-444e-ad62-e9a49bcd2fd1 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:20.841348 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1055/2117] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:47:20 2022] PUT /identity/v3/projects/fcf58f51026c4ee2bb9d8d9452208dce/users/04dea46a9cbc49afa81bc333f34c50db/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:20.851854 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f4b6c-3042-4297-983d-8b3e76ce7ea5 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:20.852057 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f4b6c-3042-4297-983d-8b3e76ce7ea5 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:20.852319 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f4b6c-3042-4297-983d-8b3e76ce7ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:20.856876 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-c10336ae-4c67-444e-ad62-e9a49bcd2fd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0e7e5bdb6c44cdbaa611ef85f78209c', 'request_id': 'req-c10336ae-4c67-444e-ad62-e9a49bcd2fd1'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:20.865225 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-c10336ae-4c67-444e-ad62-e9a49bcd2fd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0e7e5bdb6c44cdbaa611ef85f78209c', 'request_id': 'req-c10336ae-4c67-444e-ad62-e9a49bcd2fd1'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:20.879513 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-4e1f4b6c-3042-4297-983d-8b3e76ce7ea5 None None] MFA Rules not processed for user `04dea46a9cbc49afa81bc333f34c50db`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:20.880747 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-4e1f4b6c-3042-4297-983d-8b3e76ce7ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:20.945894 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1056/2118] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:20 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:47:20.970793 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-c10336ae-4c67-444e-ad62-e9a49bcd2fd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0e7e5bdb6c44cdbaa611ef85f78209c', 'request_id': 'req-c10336ae-4c67-444e-ad62-e9a49bcd2fd1'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:20.982258 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-c10336ae-4c67-444e-ad62-e9a49bcd2fd1 admin admin] The token cache is being invalidate because project d0e7e5bdb6c44cdbaa611ef85f78209c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:47:20.982567 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-c10336ae-4c67-444e-ad62-e9a49bcd2fd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:20.983233 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-c10336ae-4c67-444e-ad62-e9a49bcd2fd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:20.984819 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1063/2119] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:47:20 2022] DELETE /identity/v3/projects/d0e7e5bdb6c44cdbaa611ef85f78209c => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:20.988737 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1f4b6c-3042-4297-983d-8b3e76ce7ea5 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:21.052310 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1f4b6c-3042-4297-983d-8b3e76ce7ea5 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:21.057672 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6055d0aa-ba1b-4041-8473-df937c1177b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:21.058612 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6055d0aa-ba1b-4041-8473-df937c1177b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:21.058892 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6055d0aa-ba1b-4041-8473-df937c1177b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:21.059163 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6055d0aa-ba1b-4041-8473-df937c1177b8 admin admin] PATH_INFO: `/v3/users/948d7c5da7254ed2840787c8a199105a` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:21.060458 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6055d0aa-ba1b-4041-8473-df937c1177b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=948d7c5da7254ed2840787c8a199105a)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:21.062449 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6055d0aa-ba1b-4041-8473-df937c1177b8 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:21.071439 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:21.201058 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-6055d0aa-ba1b-4041-8473-df937c1177b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '948d7c5da7254ed2840787c8a199105a'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:21.296949 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-6055d0aa-ba1b-4041-8473-df937c1177b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '948d7c5da7254ed2840787c8a199105a'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:21.304437 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-6055d0aa-ba1b-4041-8473-df937c1177b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '948d7c5da7254ed2840787c8a199105a'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:21.316892 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-6055d0aa-ba1b-4041-8473-df937c1177b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '948d7c5da7254ed2840787c8a199105a'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:21.317598 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-6055d0aa-ba1b-4041-8473-df937c1177b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '948d7c5da7254ed2840787c8a199105a'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:21.324785 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1057/2120] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:47:20 2022] DELETE /identity/v3/users/948d7c5da7254ed2840787c8a199105a => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:21.365625 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:21.764691 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10336ae-4c67-444e-ad62-e9a49bcd2fd1 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:21.766136 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-c10336ae-4c67-444e-ad62-e9a49bcd2fd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:21.782635 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10336ae-4c67-444e-ad62-e9a49bcd2fd1 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:21.844493 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac87c6-8b80-4c2e-b8e0-93a45e2b78a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:21.845360 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac87c6-8b80-4c2e-b8e0-93a45e2b78a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:21.845642 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac87c6-8b80-4c2e-b8e0-93a45e2b78a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:21.845919 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac87c6-8b80-4c2e-b8e0-93a45e2b78a7 admin admin] PATH_INFO: `/v3/projects/f00284e2088c48b492bb6eb49cfcac49` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:21.847121 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ac87c6-8b80-4c2e-b8e0-93a45e2b78a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f00284e2088c48b492bb6eb49cfcac49)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:21.849056 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ac87c6-8b80-4c2e-b8e0-93a45e2b78a7 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:21.857949 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:22.006876 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-97ac87c6-8b80-4c2e-b8e0-93a45e2b78a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f00284e2088c48b492bb6eb49cfcac49', 'request_id': 'req-97ac87c6-8b80-4c2e-b8e0-93a45e2b78a7'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:22.013972 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-97ac87c6-8b80-4c2e-b8e0-93a45e2b78a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f00284e2088c48b492bb6eb49cfcac49', 'request_id': 'req-97ac87c6-8b80-4c2e-b8e0-93a45e2b78a7'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:22.090897 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-97ac87c6-8b80-4c2e-b8e0-93a45e2b78a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f00284e2088c48b492bb6eb49cfcac49', 'request_id': 'req-97ac87c6-8b80-4c2e-b8e0-93a45e2b78a7'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:22.101939 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-97ac87c6-8b80-4c2e-b8e0-93a45e2b78a7 admin admin] The token cache is being invalidate because project f00284e2088c48b492bb6eb49cfcac49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:47:22.102229 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-97ac87c6-8b80-4c2e-b8e0-93a45e2b78a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:22.103383 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-97ac87c6-8b80-4c2e-b8e0-93a45e2b78a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:22.104986 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1064/2121] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:47:21 2022] DELETE /identity/v3/projects/f00284e2088c48b492bb6eb49cfcac49 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:22.110063 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61f1e67-6d37-49d5-a763-c3a763185a33 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:22.110578 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61f1e67-6d37-49d5-a763-c3a763185a33 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:22.110674 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61f1e67-6d37-49d5-a763-c3a763185a33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:22.130972 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-a61f1e67-6d37-49d5-a763-c3a763185a33 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:22.136874 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a61f1e67-6d37-49d5-a763-c3a763185a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:22.195395 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1058/2122] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:47:22.203988 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c04fed-80a5-4b72-817e-b66611548c4d None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:22.211775 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c04fed-80a5-4b72-817e-b66611548c4d None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:22.211775 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c04fed-80a5-4b72-817e-b66611548c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:22.225736 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-f6c04fed-80a5-4b72-817e-b66611548c4d None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:22.227047 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-f6c04fed-80a5-4b72-817e-b66611548c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:22.236859 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1065/2123] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:22.241345 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61f1e67-6d37-49d5-a763-c3a763185a33 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:22.254683 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61f1e67-6d37-49d5-a763-c3a763185a33 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:22.261235 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf851669-218f-4aa2-95e0-171e7c7e4476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:22.261515 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf851669-218f-4aa2-95e0-171e7c7e4476 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:22.261671 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf851669-218f-4aa2-95e0-171e7c7e4476 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:22.261860 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf851669-218f-4aa2-95e0-171e7c7e4476 admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:22.262572 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf851669-218f-4aa2-95e0-171e7c7e4476 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:47:22.262876 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf851669-218f-4aa2-95e0-171e7c7e4476 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:22.264545 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf851669-218f-4aa2-95e0-171e7c7e4476 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:22.277008 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1059/2124] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 16:47:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:47:22.284660 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c04fed-80a5-4b72-817e-b66611548c4d None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:22.297274 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c04fed-80a5-4b72-817e-b66611548c4d None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:22.311794 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5a042c-c558-4812-aabd-2e82a62124c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:22.312480 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a042c-c558-4812-aabd-2e82a62124c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:22.312763 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a042c-c558-4812-aabd-2e82a62124c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:22.313042 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a042c-c558-4812-aabd-2e82a62124c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:22.313768 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5a042c-c558-4812-aabd-2e82a62124c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:22.315830 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5a042c-c558-4812-aabd-2e82a62124c9 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:22.407984 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1066/2125] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:47:22 2022] POST /identity/v3/projects => generated 365 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:47:22.412738 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf851669-218f-4aa2-95e0-171e7c7e4476 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:22.423700 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf851669-218f-4aa2-95e0-171e7c7e4476 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:22.480923 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e118222-82a1-46b6-9b1e-edeb1cc0143d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:22.481444 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e118222-82a1-46b6-9b1e-edeb1cc0143d admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:22.481726 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e118222-82a1-46b6-9b1e-edeb1cc0143d admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:22.481999 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e118222-82a1-46b6-9b1e-edeb1cc0143d admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:22.482715 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e118222-82a1-46b6-9b1e-edeb1cc0143d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:22.484751 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e118222-82a1-46b6-9b1e-edeb1cc0143d admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:22.590719 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1060/2126] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:47:22 2022] POST /identity/v3/users => generated 283 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:47:22.594873 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5a042c-c558-4812-aabd-2e82a62124c9 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:22.606357 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5a042c-c558-4812-aabd-2e82a62124c9 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:22.611584 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb377c1-112f-49c7-a8e1-731462943881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:22.612274 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb377c1-112f-49c7-a8e1-731462943881 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:22.612556 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb377c1-112f-49c7-a8e1-731462943881 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:22.612831 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb377c1-112f-49c7-a8e1-731462943881 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:22.613568 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb377c1-112f-49c7-a8e1-731462943881 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:22.615244 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb377c1-112f-49c7-a8e1-731462943881 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:22.630180 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1067/2127] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:47:22.631484 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e118222-82a1-46b6-9b1e-edeb1cc0143d admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:22.642328 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e118222-82a1-46b6-9b1e-edeb1cc0143d admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:22.647319 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7fcd8d-e654-4f26-a39a-d30a211741fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:22.648043 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7fcd8d-e654-4f26-a39a-d30a211741fb admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:22.648333 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7fcd8d-e654-4f26-a39a-d30a211741fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:22.648616 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7fcd8d-e654-4f26-a39a-d30a211741fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:22.649388 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7fcd8d-e654-4f26-a39a-d30a211741fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:22.651600 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7fcd8d-e654-4f26-a39a-d30a211741fb admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:22.661991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1061/2128] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:47:22.666986 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb377c1-112f-49c7-a8e1-731462943881 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:22.677161 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb377c1-112f-49c7-a8e1-731462943881 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:22.682212 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01681932-d69b-4697-a33b-469cfd2d29a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:22.683129 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01681932-d69b-4697-a33b-469cfd2d29a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:22.683565 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01681932-d69b-4697-a33b-469cfd2d29a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:22.683849 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01681932-d69b-4697-a33b-469cfd2d29a0 admin admin] PATH_INFO: `/v3/projects/155f6af99ce44b478f2d047ec19ed191/users/7555565300b841908d2a1797e8e5e65f/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:22.700354 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01681932-d69b-4697-a33b-469cfd2d29a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=155f6af99ce44b478f2d047ec19ed191, user_id=7555565300b841908d2a1797e8e5e65f, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:22.703178 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01681932-d69b-4697-a33b-469cfd2d29a0 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:22.746488 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:22.786328 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1068/2129] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:47:22 2022] PUT /identity/v3/projects/155f6af99ce44b478f2d047ec19ed191/users/7555565300b841908d2a1797e8e5e65f/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:47:22.797559 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182eba4f-fc69-4e11-8b79-c0d20697b240 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:22.797850 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182eba4f-fc69-4e11-8b79-c0d20697b240 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:22.798124 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182eba4f-fc69-4e11-8b79-c0d20697b240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:22.828574 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-182eba4f-fc69-4e11-8b79-c0d20697b240 None None] MFA Rules not processed for user `7555565300b841908d2a1797e8e5e65f`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:22.829941 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-182eba4f-fc69-4e11-8b79-c0d20697b240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:22.845696 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:22.900034 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1062/2130] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:22 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:22.905182 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01681932-d69b-4697-a33b-469cfd2d29a0 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:22.972526 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01681932-d69b-4697-a33b-469cfd2d29a0 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:22.977362 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50061a2-2474-4b84-93c3-76f09f6a510f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:22.982881 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50061a2-2474-4b84-93c3-76f09f6a510f admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:22.982881 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50061a2-2474-4b84-93c3-76f09f6a510f admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:22.982881 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50061a2-2474-4b84-93c3-76f09f6a510f admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:22.982881 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50061a2-2474-4b84-93c3-76f09f6a510f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:22.982881 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50061a2-2474-4b84-93c3-76f09f6a510f admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:23.099536 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1069/2131] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:47:22 2022] POST /identity/v3/projects => generated 365 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:47:23.103421 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182eba4f-fc69-4e11-8b79-c0d20697b240 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:23.114729 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182eba4f-fc69-4e11-8b79-c0d20697b240 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:23.185429 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57956ffd-9a4f-4776-a64f-dc144794da33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:23.186116 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57956ffd-9a4f-4776-a64f-dc144794da33 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:23.186443 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57956ffd-9a4f-4776-a64f-dc144794da33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:23.186723 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57956ffd-9a4f-4776-a64f-dc144794da33 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:23.187435 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57956ffd-9a4f-4776-a64f-dc144794da33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:23.189401 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57956ffd-9a4f-4776-a64f-dc144794da33 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:23.229818 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50061a2-2474-4b84-93c3-76f09f6a510f admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:23.231962 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-c50061a2-2474-4b84-93c3-76f09f6a510f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:23.303390 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1063/2132] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:47:23 2022] POST /identity/v3/users => generated 289 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:47:23.303712 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50061a2-2474-4b84-93c3-76f09f6a510f admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:23.308260 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57956ffd-9a4f-4776-a64f-dc144794da33 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:23.308400 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0628b4e2-a7ae-462b-b93a-8cffe78fbbe1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:23.309057 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0628b4e2-a7ae-462b-b93a-8cffe78fbbe1 service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:23.309354 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0628b4e2-a7ae-462b-b93a-8cffe78fbbe1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:23.309632 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0628b4e2-a7ae-462b-b93a-8cffe78fbbe1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:23.311509 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-0628b4e2-a7ae-462b-b93a-8cffe78fbbe1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:23.338298 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57956ffd-9a4f-4776-a64f-dc144794da33 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:23.343484 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabc2319-56c0-4f24-85d3-926caa588277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:23.344185 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc2319-56c0-4f24-85d3-926caa588277 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:23.344478 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc2319-56c0-4f24-85d3-926caa588277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:23.344761 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc2319-56c0-4f24-85d3-926caa588277 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:23.345498 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabc2319-56c0-4f24-85d3-926caa588277 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:23.348005 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabc2319-56c0-4f24-85d3-926caa588277 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:23.365131 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1064/2133] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:23 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:47:23.366993 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabc2319-56c0-4f24-85d3-926caa588277 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:23.378734 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabc2319-56c0-4f24-85d3-926caa588277 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:23.383724 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da4d7a-f118-4e19-970e-44b16d41c3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:23.385224 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da4d7a-f118-4e19-970e-44b16d41c3e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:23.385697 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da4d7a-f118-4e19-970e-44b16d41c3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:23.385979 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da4d7a-f118-4e19-970e-44b16d41c3e9 admin admin] PATH_INFO: `/v3/projects/303a83090d7f48bf8e0b50924153bda3/users/91e995caa20443a4abffd947255771f8/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:23.388510 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0628b4e2-a7ae-462b-b93a-8cffe78fbbe1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:23.390229 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0628b4e2-a7ae-462b-b93a-8cffe78fbbe1 service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:23.402438 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54da4d7a-f118-4e19-970e-44b16d41c3e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=303a83090d7f48bf8e0b50924153bda3, user_id=91e995caa20443a4abffd947255771f8, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:23.403365 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1070/2134] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:23 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:47:23.405375 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54da4d7a-f118-4e19-970e-44b16d41c3e9 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:23.491742 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1065/2135] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:47:23 2022] PUT /identity/v3/projects/303a83090d7f48bf8e0b50924153bda3/users/91e995caa20443a4abffd947255771f8/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:47:23.496194 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0628b4e2-a7ae-462b-b93a-8cffe78fbbe1 service cinder] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:23.506746 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0628b4e2-a7ae-462b-b93a-8cffe78fbbe1 service cinder] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:23.581904 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90795a5c-09b0-4172-bbce-15802a61b64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:23.591441 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90795a5c-09b0-4172-bbce-15802a61b64e admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:23.591695 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:23.591820 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90795a5c-09b0-4172-bbce-15802a61b64e admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:23.591820 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90795a5c-09b0-4172-bbce-15802a61b64e admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:23.591820 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90795a5c-09b0-4172-bbce-15802a61b64e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:23.591820 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90795a5c-09b0-4172-bbce-15802a61b64e admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:23.601283 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1071/2136] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:23 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:47:23.602456 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da4d7a-f118-4e19-970e-44b16d41c3e9 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:23.613637 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da4d7a-f118-4e19-970e-44b16d41c3e9 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:23.618801 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc2ba14-3cf2-4e2f-9836-6a8fa98f4851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:23.619816 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc2ba14-3cf2-4e2f-9836-6a8fa98f4851 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:23.620104 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc2ba14-3cf2-4e2f-9836-6a8fa98f4851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:23.620377 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc2ba14-3cf2-4e2f-9836-6a8fa98f4851 admin admin] PATH_INFO: `/v3/projects/303a83090d7f48bf8e0b50924153bda3/users/91e995caa20443a4abffd947255771f8/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:23.622572 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc2ba14-3cf2-4e2f-9836-6a8fa98f4851 admin admin] RBAC: Authorizing `identity:create_grant(project_id=303a83090d7f48bf8e0b50924153bda3, user_id=91e995caa20443a4abffd947255771f8, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:23.625725 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc2ba14-3cf2-4e2f-9836-6a8fa98f4851 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:23.640211 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1066/2137] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:47:23 2022] PUT /identity/v3/projects/303a83090d7f48bf8e0b50924153bda3/users/91e995caa20443a4abffd947255771f8/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:47:23.651991 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586435fc-1a17-4a85-836e-07aec9cee482 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:23.652284 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586435fc-1a17-4a85-836e-07aec9cee482 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:23.653058 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586435fc-1a17-4a85-836e-07aec9cee482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:23.682041 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-586435fc-1a17-4a85-836e-07aec9cee482 None None] MFA Rules not processed for user `91e995caa20443a4abffd947255771f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:23.683241 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-586435fc-1a17-4a85-836e-07aec9cee482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:23.765168 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1072/2138] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:23 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:23.777728 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc2ba14-3cf2-4e2f-9836-6a8fa98f4851 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:23.780660 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-ffc2ba14-3cf2-4e2f-9836-6a8fa98f4851 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:23.848142 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc2ba14-3cf2-4e2f-9836-6a8fa98f4851 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:23.851970 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea350c0-8d75-4790-88d5-8c71389ad26d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:23.852699 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea350c0-8d75-4790-88d5-8c71389ad26d service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:23.853443 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea350c0-8d75-4790-88d5-8c71389ad26d service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:23.853693 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea350c0-8d75-4790-88d5-8c71389ad26d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:23.863764 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea350c0-8d75-4790-88d5-8c71389ad26d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:23.865226 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea350c0-8d75-4790-88d5-8c71389ad26d service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:23.875743 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1067/2139] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:23 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:47:24.116638 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586435fc-1a17-4a85-836e-07aec9cee482 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:24.127305 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586435fc-1a17-4a85-836e-07aec9cee482 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:24.131644 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fecea7-0427-49cd-97b1-df6da6af8400 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:24.132314 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fecea7-0427-49cd-97b1-df6da6af8400 service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:24.132601 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fecea7-0427-49cd-97b1-df6da6af8400 service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:24.132879 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fecea7-0427-49cd-97b1-df6da6af8400 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:24.194492 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fecea7-0427-49cd-97b1-df6da6af8400 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:24.196046 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fecea7-0427-49cd-97b1-df6da6af8400 service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:24.207189 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1073/2140] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:24 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:47:24.348482 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:24.873812 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:25.060330 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:25.117534 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:25.445536 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:25.485617 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:25.828612 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:26.222506 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:26.746209 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:26.786276 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:27.209053 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:27.296974 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=88,keystone:DELETE=8,keystone:INSERT=4 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:27.572177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:27.668360 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:28.044780 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea350c0-8d75-4790-88d5-8c71389ad26d service cinder] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:28.112494 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea350c0-8d75-4790-88d5-8c71389ad26d service cinder] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:28.117559 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad570e6a-cd20-4760-b715-99b96de74660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:28.118444 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad570e6a-cd20-4760-b715-99b96de74660 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:28.118733 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad570e6a-cd20-4760-b715-99b96de74660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:28.119017 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad570e6a-cd20-4760-b715-99b96de74660 admin admin] PATH_INFO: `/v3/users/7555565300b841908d2a1797e8e5e65f` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:28.120248 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad570e6a-cd20-4760-b715-99b96de74660 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7555565300b841908d2a1797e8e5e65f)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:28.122183 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad570e6a-cd20-4760-b715-99b96de74660 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:28.154323 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:28.339562 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-ad570e6a-cd20-4760-b715-99b96de74660 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7555565300b841908d2a1797e8e5e65f'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:28.348469 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:28.408793 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-ad570e6a-cd20-4760-b715-99b96de74660 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7555565300b841908d2a1797e8e5e65f'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:28.417066 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-ad570e6a-cd20-4760-b715-99b96de74660 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7555565300b841908d2a1797e8e5e65f'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:28.428743 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-ad570e6a-cd20-4760-b715-99b96de74660 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7555565300b841908d2a1797e8e5e65f'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:28.429424 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-ad570e6a-cd20-4760-b715-99b96de74660 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7555565300b841908d2a1797e8e5e65f'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:28.435446 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1068/2141] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:47:28 2022] DELETE /identity/v3/users/7555565300b841908d2a1797e8e5e65f => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:28.440088 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea0679-48c6-4b2f-9287-8a7c0d6da9aa None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:28.440387 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea0679-48c6-4b2f-9287-8a7c0d6da9aa None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:28.440673 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea0679-48c6-4b2f-9287-8a7c0d6da9aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:28.458577 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-91ea0679-48c6-4b2f-9287-8a7c0d6da9aa None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:28.459844 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-91ea0679-48c6-4b2f-9287-8a7c0d6da9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:28.525356 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1074/2142] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:28.535290 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad570e6a-cd20-4760-b715-99b96de74660 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:28.536748 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-ad570e6a-cd20-4760-b715-99b96de74660 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:28.600490 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad570e6a-cd20-4760-b715-99b96de74660 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:28.605097 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935a9229-91be-43cb-be4f-e1d65b10047a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:28.605781 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a9229-91be-43cb-be4f-e1d65b10047a service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:28.606066 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a9229-91be-43cb-be4f-e1d65b10047a service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:28.606343 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a9229-91be-43cb-be4f-e1d65b10047a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:28.616990 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935a9229-91be-43cb-be4f-e1d65b10047a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:28.618394 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935a9229-91be-43cb-be4f-e1d65b10047a service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:28.628661 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1069/2143] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:47:28.697549 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:28.797429 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:29.109163 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea0679-48c6-4b2f-9287-8a7c0d6da9aa None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:29.110693 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-91ea0679-48c6-4b2f-9287-8a7c0d6da9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:29.125719 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea0679-48c6-4b2f-9287-8a7c0d6da9aa None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:29.130792 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe79ed-3e6c-482c-a2fa-be829442836b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:29.131732 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe79ed-3e6c-482c-a2fa-be829442836b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:29.132024 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe79ed-3e6c-482c-a2fa-be829442836b admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:29.132303 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe79ed-3e6c-482c-a2fa-be829442836b admin admin] PATH_INFO: `/v3/projects/155f6af99ce44b478f2d047ec19ed191` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:29.135570 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfe79ed-3e6c-482c-a2fa-be829442836b admin admin] RBAC: Authorizing `identity:delete_project(project_id=155f6af99ce44b478f2d047ec19ed191)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:29.138789 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfe79ed-3e6c-482c-a2fa-be829442836b admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:29.339110 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-7cfe79ed-3e6c-482c-a2fa-be829442836b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '155f6af99ce44b478f2d047ec19ed191', 'request_id': 'req-7cfe79ed-3e6c-482c-a2fa-be829442836b'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:29.346135 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-7cfe79ed-3e6c-482c-a2fa-be829442836b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '155f6af99ce44b478f2d047ec19ed191', 'request_id': 'req-7cfe79ed-3e6c-482c-a2fa-be829442836b'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:29.433932 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:29.451020 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-7cfe79ed-3e6c-482c-a2fa-be829442836b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '155f6af99ce44b478f2d047ec19ed191', 'request_id': 'req-7cfe79ed-3e6c-482c-a2fa-be829442836b'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:29.463446 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-7cfe79ed-3e6c-482c-a2fa-be829442836b admin admin] The token cache is being invalidate because project 155f6af99ce44b478f2d047ec19ed191 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:47:29.463819 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-7cfe79ed-3e6c-482c-a2fa-be829442836b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:29.464629 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-7cfe79ed-3e6c-482c-a2fa-be829442836b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:29.466319 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1075/2144] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:47:29 2022] DELETE /identity/v3/projects/155f6af99ce44b478f2d047ec19ed191 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:29.475354 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935a9229-91be-43cb-be4f-e1d65b10047a service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:29.489633 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935a9229-91be-43cb-be4f-e1d65b10047a service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:29.492289 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:29.551608 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcc1662-58c4-42b9-8732-becab4a0a417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:29.554722 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcc1662-58c4-42b9-8732-becab4a0a417 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:29.555008 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcc1662-58c4-42b9-8732-becab4a0a417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:29.555288 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcc1662-58c4-42b9-8732-becab4a0a417 admin admin] PATH_INFO: `/v3/users/91e995caa20443a4abffd947255771f8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:29.557193 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcc1662-58c4-42b9-8732-becab4a0a417 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91e995caa20443a4abffd947255771f8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:29.559679 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcc1662-58c4-42b9-8732-becab4a0a417 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:29.757368 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-bfcc1662-58c4-42b9-8732-becab4a0a417 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91e995caa20443a4abffd947255771f8'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:29.878160 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:29.878301 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-bfcc1662-58c4-42b9-8732-becab4a0a417 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91e995caa20443a4abffd947255771f8'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:29.887465 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-bfcc1662-58c4-42b9-8732-becab4a0a417 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91e995caa20443a4abffd947255771f8'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:29.904323 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-bfcc1662-58c4-42b9-8732-becab4a0a417 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91e995caa20443a4abffd947255771f8'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:29.905518 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-bfcc1662-58c4-42b9-8732-becab4a0a417 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91e995caa20443a4abffd947255771f8'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:29.914459 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1070/2145] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:47:29 2022] DELETE /identity/v3/users/91e995caa20443a4abffd947255771f8 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:30.011917 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:30.449790 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe79ed-3e6c-482c-a2fa-be829442836b admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:30.451682 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-7cfe79ed-3e6c-482c-a2fa-be829442836b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:30.466146 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe79ed-3e6c-482c-a2fa-be829442836b admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:30.545287 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a0d66-a015-4c33-9bcc-7eee62d15a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:30.546678 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a0d66-a015-4c33-9bcc-7eee62d15a32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:30.547685 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a0d66-a015-4c33-9bcc-7eee62d15a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:30.547969 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a0d66-a015-4c33-9bcc-7eee62d15a32 admin admin] PATH_INFO: `/v3/projects/303a83090d7f48bf8e0b50924153bda3` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:30.549243 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180a0d66-a015-4c33-9bcc-7eee62d15a32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=303a83090d7f48bf8e0b50924153bda3)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:30.551207 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180a0d66-a015-4c33-9bcc-7eee62d15a32 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:30.724870 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-180a0d66-a015-4c33-9bcc-7eee62d15a32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '303a83090d7f48bf8e0b50924153bda3', 'request_id': 'req-180a0d66-a015-4c33-9bcc-7eee62d15a32'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:30.732730 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-180a0d66-a015-4c33-9bcc-7eee62d15a32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '303a83090d7f48bf8e0b50924153bda3', 'request_id': 'req-180a0d66-a015-4c33-9bcc-7eee62d15a32'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:30.886551 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-180a0d66-a015-4c33-9bcc-7eee62d15a32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '303a83090d7f48bf8e0b50924153bda3', 'request_id': 'req-180a0d66-a015-4c33-9bcc-7eee62d15a32'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:30.901806 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-180a0d66-a015-4c33-9bcc-7eee62d15a32 admin admin] The token cache is being invalidate because project 303a83090d7f48bf8e0b50924153bda3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:47:30.902615 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-180a0d66-a015-4c33-9bcc-7eee62d15a32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:30.903364 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-180a0d66-a015-4c33-9bcc-7eee62d15a32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:30.905137 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1076/2146] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:47:30 2022] DELETE /identity/v3/projects/303a83090d7f48bf8e0b50924153bda3 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:30.917752 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd6825-384c-4840-abc5-cf65d606420a None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:30.918327 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd6825-384c-4840-abc5-cf65d606420a None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:30.918671 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd6825-384c-4840-abc5-cf65d606420a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:30.942033 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-52fd6825-384c-4840-abc5-cf65d606420a None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:30.943637 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-52fd6825-384c-4840-abc5-cf65d606420a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:31.018234 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1071/2147] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:47:31.027042 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb468a1-e5cb-4f23-bc7b-6da64dd0adb7 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:31.027708 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb468a1-e5cb-4f23-bc7b-6da64dd0adb7 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:31.028107 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb468a1-e5cb-4f23-bc7b-6da64dd0adb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:31.051742 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-fcb468a1-e5cb-4f23-bc7b-6da64dd0adb7 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:31.052979 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-fcb468a1-e5cb-4f23-bc7b-6da64dd0adb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:31.062361 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1077/2148] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:31.066713 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fd6825-384c-4840-abc5-cf65d606420a None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:31.079651 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fd6825-384c-4840-abc5-cf65d606420a None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:31.084806 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b237ca-a084-4388-9631-bc91ea5c1490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:31.085677 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b237ca-a084-4388-9631-bc91ea5c1490 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:31.086133 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b237ca-a084-4388-9631-bc91ea5c1490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:31.086416 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b237ca-a084-4388-9631-bc91ea5c1490 admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:31.087150 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b237ca-a084-4388-9631-bc91ea5c1490 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:47:31.088266 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b237ca-a084-4388-9631-bc91ea5c1490 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:31.090850 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b237ca-a084-4388-9631-bc91ea5c1490 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:31.104229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1072/2149] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 16:47:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:47:31.110045 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb468a1-e5cb-4f23-bc7b-6da64dd0adb7 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:31.121398 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb468a1-e5cb-4f23-bc7b-6da64dd0adb7 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:31.126708 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f79d9a7-07ad-4740-b1c0-9dd19547b820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:31.127640 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f79d9a7-07ad-4740-b1c0-9dd19547b820 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:31.127924 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f79d9a7-07ad-4740-b1c0-9dd19547b820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:31.128255 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f79d9a7-07ad-4740-b1c0-9dd19547b820 admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:31.129057 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f79d9a7-07ad-4740-b1c0-9dd19547b820 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:31.131097 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f79d9a7-07ad-4740-b1c0-9dd19547b820 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:31.237240 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1078/2150] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:47:31 2022] POST /identity/v3/projects => generated 379 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:47:31.241070 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b237ca-a084-4388-9631-bc91ea5c1490 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:31.253454 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b237ca-a084-4388-9631-bc91ea5c1490 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:31.276909 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f79d9a7-07ad-4740-b1c0-9dd19547b820 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:31.279567 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-3f79d9a7-07ad-4740-b1c0-9dd19547b820 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:31.316461 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842507b1-3904-4c39-9ac9-41eb71c3e584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:31.317032 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842507b1-3904-4c39-9ac9-41eb71c3e584 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:31.317370 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842507b1-3904-4c39-9ac9-41eb71c3e584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:31.317681 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842507b1-3904-4c39-9ac9-41eb71c3e584 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:31.318417 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842507b1-3904-4c39-9ac9-41eb71c3e584 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:31.320864 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842507b1-3904-4c39-9ac9-41eb71c3e584 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:31.351915 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f79d9a7-07ad-4740-b1c0-9dd19547b820 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:31.357113 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66accec1-1c81-4d23-ad03-096b3d330b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:31.358089 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66accec1-1c81-4d23-ad03-096b3d330b30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:31.358374 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66accec1-1c81-4d23-ad03-096b3d330b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:31.358654 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66accec1-1c81-4d23-ad03-096b3d330b30 admin admin] PATH_INFO: `/v3/users/7fb5d5bca45845de8c487e3dca445e22` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:31.359915 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66accec1-1c81-4d23-ad03-096b3d330b30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fb5d5bca45845de8c487e3dca445e22)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:31.361847 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66accec1-1c81-4d23-ad03-096b3d330b30 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:31.432474 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1073/2151] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:47:31 2022] POST /identity/v3/users => generated 290 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:47:31.436547 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842507b1-3904-4c39-9ac9-41eb71c3e584 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:31.446188 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842507b1-3904-4c39-9ac9-41eb71c3e584 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:31.452251 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370c771b-9517-40e9-9c50-091cd3835924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:31.452913 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370c771b-9517-40e9-9c50-091cd3835924 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:31.453202 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370c771b-9517-40e9-9c50-091cd3835924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:31.453487 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370c771b-9517-40e9-9c50-091cd3835924 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:31.454265 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370c771b-9517-40e9-9c50-091cd3835924 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:31.455934 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370c771b-9517-40e9-9c50-091cd3835924 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:31.466308 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1074/2152] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:47:31.470346 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370c771b-9517-40e9-9c50-091cd3835924 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:31.476550 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:31.482557 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370c771b-9517-40e9-9c50-091cd3835924 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:31.487350 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee19534-c768-443c-8aba-2a3d0bf8f490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:31.488089 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee19534-c768-443c-8aba-2a3d0bf8f490 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:31.488373 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee19534-c768-443c-8aba-2a3d0bf8f490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:31.488826 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee19534-c768-443c-8aba-2a3d0bf8f490 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:31.489652 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee19534-c768-443c-8aba-2a3d0bf8f490 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:31.491308 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee19534-c768-443c-8aba-2a3d0bf8f490 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:31.503552 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1075/2153] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:47:31.507640 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee19534-c768-443c-8aba-2a3d0bf8f490 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:31.517192 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee19534-c768-443c-8aba-2a3d0bf8f490 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:31.522443 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb2c9ba-7d78-4617-a1be-a78e59b8a491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:31.523365 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2c9ba-7d78-4617-a1be-a78e59b8a491 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:31.523677 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2c9ba-7d78-4617-a1be-a78e59b8a491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:31.523979 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2c9ba-7d78-4617-a1be-a78e59b8a491 admin admin] PATH_INFO: `/v3/projects/ebf10aef0e65482dbba183d25f564bbe/users/8947243c85c34979b6aa821f741d89e8/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:31.538854 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb2c9ba-7d78-4617-a1be-a78e59b8a491 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebf10aef0e65482dbba183d25f564bbe, user_id=8947243c85c34979b6aa821f741d89e8, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:31.542050 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb2c9ba-7d78-4617-a1be-a78e59b8a491 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:31.592940 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-66accec1-1c81-4d23-ad03-096b3d330b30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fb5d5bca45845de8c487e3dca445e22'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:31.659228 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1076/2154] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:47:31 2022] PUT /identity/v3/projects/ebf10aef0e65482dbba183d25f564bbe/users/8947243c85c34979b6aa821f741d89e8/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:47:31.670567 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55feb889-dbfd-4da2-87e4-bdaa34efa88d None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:31.670852 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55feb889-dbfd-4da2-87e4-bdaa34efa88d None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:31.671135 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55feb889-dbfd-4da2-87e4-bdaa34efa88d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:31.686084 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-66accec1-1c81-4d23-ad03-096b3d330b30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fb5d5bca45845de8c487e3dca445e22'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:31.686836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-66accec1-1c81-4d23-ad03-096b3d330b30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fb5d5bca45845de8c487e3dca445e22'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:31.687483 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-66accec1-1c81-4d23-ad03-096b3d330b30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fb5d5bca45845de8c487e3dca445e22'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:31.706419 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-66accec1-1c81-4d23-ad03-096b3d330b30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fb5d5bca45845de8c487e3dca445e22'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:31.707781 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-55feb889-dbfd-4da2-87e4-bdaa34efa88d None None] MFA Rules not processed for user `8947243c85c34979b6aa821f741d89e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:31.709646 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-55feb889-dbfd-4da2-87e4-bdaa34efa88d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:31.726694 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1079/2155] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:47:31 2022] DELETE /identity/v3/users/7fb5d5bca45845de8c487e3dca445e22 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:47:31.731424 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b44d3d3-ff7a-4e48-91ce-6807f9695ca0 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:31.731907 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b44d3d3-ff7a-4e48-91ce-6807f9695ca0 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:31.732196 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b44d3d3-ff7a-4e48-91ce-6807f9695ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:31.750263 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-2b44d3d3-ff7a-4e48-91ce-6807f9695ca0 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:31.751519 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-2b44d3d3-ff7a-4e48-91ce-6807f9695ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:31.789187 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1077/2156] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:47:31.794443 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55feb889-dbfd-4da2-87e4-bdaa34efa88d None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:31.795907 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-55feb889-dbfd-4da2-87e4-bdaa34efa88d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:31.803748 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:31.830544 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1080/2157] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:31.843007 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b44d3d3-ff7a-4e48-91ce-6807f9695ca0 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:31.845112 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-2b44d3d3-ff7a-4e48-91ce-6807f9695ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:31.870523 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55feb889-dbfd-4da2-87e4-bdaa34efa88d None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:31.879881 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc570507-458e-489d-8e0c-0fdac7b7e82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:31.880688 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc570507-458e-489d-8e0c-0fdac7b7e82b admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:31.880973 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc570507-458e-489d-8e0c-0fdac7b7e82b admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:31.881321 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc570507-458e-489d-8e0c-0fdac7b7e82b admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:31.882233 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc570507-458e-489d-8e0c-0fdac7b7e82b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:31.884315 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc570507-458e-489d-8e0c-0fdac7b7e82b admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:31.916609 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b44d3d3-ff7a-4e48-91ce-6807f9695ca0 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:31.920467 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8572fea8-6cc6-4f36-af68-5f43a759f24f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:31.921134 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8572fea8-6cc6-4f36-af68-5f43a759f24f service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:31.921420 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8572fea8-6cc6-4f36-af68-5f43a759f24f service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:31.921698 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8572fea8-6cc6-4f36-af68-5f43a759f24f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:31.931424 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8572fea8-6cc6-4f36-af68-5f43a759f24f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:31.933080 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8572fea8-6cc6-4f36-af68-5f43a759f24f service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:31.943840 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1081/2158] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:47:31.971058 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1078/2159] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:47:31 2022] POST /identity/v3/projects => generated 381 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:47:31.981740 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8572fea8-6cc6-4f36-af68-5f43a759f24f service neutron] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:31.991875 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8572fea8-6cc6-4f36-af68-5f43a759f24f service neutron] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:32.054202 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bde240-598d-4d11-8eb7-463861ba5b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:32.054912 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bde240-598d-4d11-8eb7-463861ba5b0f admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:32.055195 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bde240-598d-4d11-8eb7-463861ba5b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:32.055463 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bde240-598d-4d11-8eb7-463861ba5b0f admin admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:32.056278 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bde240-598d-4d11-8eb7-463861ba5b0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:32.058641 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bde240-598d-4d11-8eb7-463861ba5b0f admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:32.206857 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1082/2160] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:47:31 2022] POST /identity/v3/users => generated 297 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:47:32.210588 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc570507-458e-489d-8e0c-0fdac7b7e82b admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:32.221285 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc570507-458e-489d-8e0c-0fdac7b7e82b admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:32.230120 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd8d09-d574-4925-86e3-890d75776906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:32.230811 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd8d09-d574-4925-86e3-890d75776906 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:32.231100 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd8d09-d574-4925-86e3-890d75776906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:32.231382 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd8d09-d574-4925-86e3-890d75776906 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:32.232193 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acd8d09-d574-4925-86e3-890d75776906 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:32.234699 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acd8d09-d574-4925-86e3-890d75776906 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:32.247170 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1079/2161] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:47:32.253081 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bde240-598d-4d11-8eb7-463861ba5b0f admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:32.265944 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bde240-598d-4d11-8eb7-463861ba5b0f admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:32.271702 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d37ad2-c04d-4309-83b2-c8ac7d50b7ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:32.272642 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d37ad2-c04d-4309-83b2-c8ac7d50b7ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:32.272929 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d37ad2-c04d-4309-83b2-c8ac7d50b7ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:32.273212 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d37ad2-c04d-4309-83b2-c8ac7d50b7ad admin admin] PATH_INFO: `/v3/projects/764985ac006a443095664f8ec2ac4479/users/8f2eda535fe54da78f37a2b8c88cbec6/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:32.289586 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d37ad2-c04d-4309-83b2-c8ac7d50b7ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=764985ac006a443095664f8ec2ac4479, user_id=8f2eda535fe54da78f37a2b8c88cbec6, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:32.296882 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d37ad2-c04d-4309-83b2-c8ac7d50b7ad admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:32.381330 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1083/2162] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:47:32 2022] PUT /identity/v3/projects/764985ac006a443095664f8ec2ac4479/users/8f2eda535fe54da78f37a2b8c88cbec6/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:32.385597 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd8d09-d574-4925-86e3-890d75776906 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:32.396220 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd8d09-d574-4925-86e3-890d75776906 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:32.397414 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d37ad2-c04d-4309-83b2-c8ac7d50b7ad admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:32.399275 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-89d37ad2-c04d-4309-83b2-c8ac7d50b7ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:32.469079 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de21306-c210-474a-834b-9a5c50ac4b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:32.469798 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de21306-c210-474a-834b-9a5c50ac4b4a admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:32.470135 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de21306-c210-474a-834b-9a5c50ac4b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:32.470413 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de21306-c210-474a-834b-9a5c50ac4b4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:32.471155 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de21306-c210-474a-834b-9a5c50ac4b4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:32.472840 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de21306-c210-474a-834b-9a5c50ac4b4a admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:32.477625 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d37ad2-c04d-4309-83b2-c8ac7d50b7ad admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:32.483141 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3091f07-580e-4e02-b72b-a47901b75d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:32.484011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1080/2163] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:32 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:47:32.484128 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3091f07-580e-4e02-b72b-a47901b75d05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:32.484316 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3091f07-580e-4e02-b72b-a47901b75d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:32.484591 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3091f07-580e-4e02-b72b-a47901b75d05 admin admin] PATH_INFO: `/v3/projects/8482fe90b80d4758b0dec096cb492869` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:32.485854 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3091f07-580e-4e02-b72b-a47901b75d05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8482fe90b80d4758b0dec096cb492869)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:32.488179 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3091f07-580e-4e02-b72b-a47901b75d05 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:32.488720 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de21306-c210-474a-834b-9a5c50ac4b4a admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:32.499840 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de21306-c210-474a-834b-9a5c50ac4b4a admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:32.504741 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdd09f2-a78f-4f92-91c4-864d652ad06c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:32.505630 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd09f2-a78f-4f92-91c4-864d652ad06c admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:32.505914 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd09f2-a78f-4f92-91c4-864d652ad06c admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:32.506190 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd09f2-a78f-4f92-91c4-864d652ad06c admin admin] PATH_INFO: `/v3/projects/764985ac006a443095664f8ec2ac4479/users/8f2eda535fe54da78f37a2b8c88cbec6/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:32.508320 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdd09f2-a78f-4f92-91c4-864d652ad06c admin admin] RBAC: Authorizing `identity:create_grant(project_id=764985ac006a443095664f8ec2ac4479, user_id=8f2eda535fe54da78f37a2b8c88cbec6, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:32.511050 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdd09f2-a78f-4f92-91c4-864d652ad06c admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:32.525936 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1081/2164] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:47:32 2022] PUT /identity/v3/projects/764985ac006a443095664f8ec2ac4479/users/8f2eda535fe54da78f37a2b8c88cbec6/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:32.535641 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:32.537835 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04216e8-5f4c-4d89-9b46-5ee459b00e8e None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:32.538146 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04216e8-5f4c-4d89-9b46-5ee459b00e8e None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:32.538574 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04216e8-5f4c-4d89-9b46-5ee459b00e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:32.568268 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-d04216e8-5f4c-4d89-9b46-5ee459b00e8e None None] MFA Rules not processed for user `8f2eda535fe54da78f37a2b8c88cbec6`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:32.569406 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d04216e8-5f4c-4d89-9b46-5ee459b00e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:32.608075 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-d3091f07-580e-4e02-b72b-a47901b75d05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8482fe90b80d4758b0dec096cb492869', 'request_id': 'req-d3091f07-580e-4e02-b72b-a47901b75d05'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:32.614721 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-d3091f07-580e-4e02-b72b-a47901b75d05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8482fe90b80d4758b0dec096cb492869', 'request_id': 'req-d3091f07-580e-4e02-b72b-a47901b75d05'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:32.647404 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1082/2165] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:32 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:47:32.655720 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04216e8-5f4c-4d89-9b46-5ee459b00e8e None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:32.655720 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d04216e8-5f4c-4d89-9b46-5ee459b00e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:32.713067 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04216e8-5f4c-4d89-9b46-5ee459b00e8e None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:32.720103 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-d3091f07-580e-4e02-b72b-a47901b75d05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8482fe90b80d4758b0dec096cb492869', 'request_id': 'req-d3091f07-580e-4e02-b72b-a47901b75d05'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:32.732646 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-d3091f07-580e-4e02-b72b-a47901b75d05 admin admin] The token cache is being invalidate because project 8482fe90b80d4758b0dec096cb492869 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:47:32.732935 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-d3091f07-580e-4e02-b72b-a47901b75d05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:32.733624 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-d3091f07-580e-4e02-b72b-a47901b75d05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:32.751836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1084/2166] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:47:32 2022] DELETE /identity/v3/projects/8482fe90b80d4758b0dec096cb492869 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:32.752872 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3091f07-580e-4e02-b72b-a47901b75d05 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:32.776967 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-d3091f07-580e-4e02-b72b-a47901b75d05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:32.777612 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3590da4-aecf-46d8-a0f1-10f6c39aeb57 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0887908bcb0d4072bf6acb4cbaf1af83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:32.778284 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3590da4-aecf-46d8-a0f1-10f6c39aeb57 service glance] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:32.778570 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3590da4-aecf-46d8-a0f1-10f6c39aeb57 service glance] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:32.778845 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3590da4-aecf-46d8-a0f1-10f6c39aeb57 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:32.780973 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-b3590da4-aecf-46d8-a0f1-10f6c39aeb57 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:32.825209 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:32.881634 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3091f07-580e-4e02-b72b-a47901b75d05 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:32.889881 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a117f59b-009a-4223-9c1e-3bbc4dcd5162 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:32.891406 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a117f59b-009a-4223-9c1e-3bbc4dcd5162 service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:32.891856 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a117f59b-009a-4223-9c1e-3bbc4dcd5162 service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:32.892264 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a117f59b-009a-4223-9c1e-3bbc4dcd5162 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:32.895646 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-a117f59b-009a-4223-9c1e-3bbc4dcd5162 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:32.918359 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3590da4-aecf-46d8-a0f1-10f6c39aeb57 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:32.920145 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3590da4-aecf-46d8-a0f1-10f6c39aeb57 service glance] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:32.941587 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1083/2167] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:32 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:47:33.038865 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a117f59b-009a-4223-9c1e-3bbc4dcd5162 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:33.040547 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a117f59b-009a-4223-9c1e-3bbc4dcd5162 service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:33.053408 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1085/2168] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:32 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:47:33.108429 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:33.503770 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:34.138346 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:34.194661 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:34.233216 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3590da4-aecf-46d8-a0f1-10f6c39aeb57 service glance] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:34.244911 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3590da4-aecf-46d8-a0f1-10f6c39aeb57 service glance] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:34.249577 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722968da-1297-4f32-9c93-a0f99f3df11d tempest-VolumeQuotaClassesTest-1228394397 tempest-VolumeQuotaClassesTest-1228394397-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f2eda535fe54da78f37a2b8c88cbec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '764985ac006a443095664f8ec2ac4479', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:34.250268 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722968da-1297-4f32-9c93-a0f99f3df11d tempest-VolumeQuotaClassesTest-1228394397 tempest-VolumeQuotaClassesTest-1228394397-project-admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:34.250557 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722968da-1297-4f32-9c93-a0f99f3df11d tempest-VolumeQuotaClassesTest-1228394397 tempest-VolumeQuotaClassesTest-1228394397-project-admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:34.250841 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722968da-1297-4f32-9c93-a0f99f3df11d tempest-VolumeQuotaClassesTest-1228394397 tempest-VolumeQuotaClassesTest-1228394397-project-admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:34.251611 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722968da-1297-4f32-9c93-a0f99f3df11d tempest-VolumeQuotaClassesTest-1228394397 tempest-VolumeQuotaClassesTest-1228394397-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:47:34.251927 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722968da-1297-4f32-9c93-a0f99f3df11d tempest-VolumeQuotaClassesTest-1228394397 tempest-VolumeQuotaClassesTest-1228394397-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:34.253712 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722968da-1297-4f32-9c93-a0f99f3df11d tempest-VolumeQuotaClassesTest-1228394397 tempest-VolumeQuotaClassesTest-1228394397-project-admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:34.268486 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1084/2169] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 16:47:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:47:34.272775 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a117f59b-009a-4223-9c1e-3bbc4dcd5162 service cinder] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:34.283684 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a117f59b-009a-4223-9c1e-3bbc4dcd5162 service cinder] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:34.289625 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1026580-bb61-4178-a144-ca021f00f02d tempest-VolumeQuotaClassesTest-1228394397 tempest-VolumeQuotaClassesTest-1228394397-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f2eda535fe54da78f37a2b8c88cbec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '764985ac006a443095664f8ec2ac4479', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:34.290334 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1026580-bb61-4178-a144-ca021f00f02d tempest-VolumeQuotaClassesTest-1228394397 tempest-VolumeQuotaClassesTest-1228394397-project-admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:34.291534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1026580-bb61-4178-a144-ca021f00f02d tempest-VolumeQuotaClassesTest-1228394397 tempest-VolumeQuotaClassesTest-1228394397-project-admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:34.292052 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1026580-bb61-4178-a144-ca021f00f02d tempest-VolumeQuotaClassesTest-1228394397 tempest-VolumeQuotaClassesTest-1228394397-project-admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:34.293290 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1026580-bb61-4178-a144-ca021f00f02d tempest-VolumeQuotaClassesTest-1228394397 tempest-VolumeQuotaClassesTest-1228394397-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:34.295256 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1026580-bb61-4178-a144-ca021f00f02d tempest-VolumeQuotaClassesTest-1228394397 tempest-VolumeQuotaClassesTest-1228394397-project-admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:34.410462 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1086/2170] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:47:34 2022] POST /identity/v3/projects => generated 354 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:47:34.414665 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722968da-1297-4f32-9c93-a0f99f3df11d tempest-VolumeQuotaClassesTest-1228394397 tempest-VolumeQuotaClassesTest-1228394397-project-admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:34.425552 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722968da-1297-4f32-9c93-a0f99f3df11d tempest-VolumeQuotaClassesTest-1228394397 tempest-VolumeQuotaClassesTest-1228394397-project-admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:34.483434 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e72dc4-fbba-43bb-98bc-e674e445ed0e tempest-VolumeQuotaClassesTest-1228394397 tempest-VolumeQuotaClassesTest-1228394397-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f2eda535fe54da78f37a2b8c88cbec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '764985ac006a443095664f8ec2ac4479', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:34.484141 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e72dc4-fbba-43bb-98bc-e674e445ed0e tempest-VolumeQuotaClassesTest-1228394397 tempest-VolumeQuotaClassesTest-1228394397-project-admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:34.484427 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e72dc4-fbba-43bb-98bc-e674e445ed0e tempest-VolumeQuotaClassesTest-1228394397 tempest-VolumeQuotaClassesTest-1228394397-project-admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:34.484704 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e72dc4-fbba-43bb-98bc-e674e445ed0e tempest-VolumeQuotaClassesTest-1228394397 tempest-VolumeQuotaClassesTest-1228394397-project-admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:34.485431 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e72dc4-fbba-43bb-98bc-e674e445ed0e tempest-VolumeQuotaClassesTest-1228394397 tempest-VolumeQuotaClassesTest-1228394397-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:47:34.485737 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e72dc4-fbba-43bb-98bc-e674e445ed0e tempest-VolumeQuotaClassesTest-1228394397 tempest-VolumeQuotaClassesTest-1228394397-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:34.487326 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e72dc4-fbba-43bb-98bc-e674e445ed0e tempest-VolumeQuotaClassesTest-1228394397 tempest-VolumeQuotaClassesTest-1228394397-project-admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:34.498874 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1085/2171] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 16:47:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:47:34.535819 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1026580-bb61-4178-a144-ca021f00f02d tempest-VolumeQuotaClassesTest-1228394397 tempest-VolumeQuotaClassesTest-1228394397-project-admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:34.546221 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1026580-bb61-4178-a144-ca021f00f02d tempest-VolumeQuotaClassesTest-1228394397 tempest-VolumeQuotaClassesTest-1228394397-project-admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:34.550848 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1173452c-993d-4b9f-a834-60abcc979d8f tempest-VolumeQuotaClassesTest-1228394397 tempest-VolumeQuotaClassesTest-1228394397-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f2eda535fe54da78f37a2b8c88cbec6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '764985ac006a443095664f8ec2ac4479', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:34.551738 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1173452c-993d-4b9f-a834-60abcc979d8f tempest-VolumeQuotaClassesTest-1228394397 tempest-VolumeQuotaClassesTest-1228394397-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:34.552030 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1173452c-993d-4b9f-a834-60abcc979d8f tempest-VolumeQuotaClassesTest-1228394397 tempest-VolumeQuotaClassesTest-1228394397-project-admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:34.552313 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1173452c-993d-4b9f-a834-60abcc979d8f tempest-VolumeQuotaClassesTest-1228394397 tempest-VolumeQuotaClassesTest-1228394397-project-admin] PATH_INFO: `/v3/projects/f81b53e3ac924a6f8bef022378ec9e33` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:34.553510 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1173452c-993d-4b9f-a834-60abcc979d8f tempest-VolumeQuotaClassesTest-1228394397 tempest-VolumeQuotaClassesTest-1228394397-project-admin] RBAC: Authorizing `identity:delete_project(project_id=f81b53e3ac924a6f8bef022378ec9e33)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:34.555422 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1173452c-993d-4b9f-a834-60abcc979d8f tempest-VolumeQuotaClassesTest-1228394397 tempest-VolumeQuotaClassesTest-1228394397-project-admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:34.707562 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-1173452c-993d-4b9f-a834-60abcc979d8f tempest-VolumeQuotaClassesTest-1228394397 tempest-VolumeQuotaClassesTest-1228394397-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f81b53e3ac924a6f8bef022378ec9e33', 'request_id': 'req-1173452c-993d-4b9f-a834-60abcc979d8f'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:34.716132 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-1173452c-993d-4b9f-a834-60abcc979d8f tempest-VolumeQuotaClassesTest-1228394397 tempest-VolumeQuotaClassesTest-1228394397-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f81b53e3ac924a6f8bef022378ec9e33', 'request_id': 'req-1173452c-993d-4b9f-a834-60abcc979d8f'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:34.754415 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:34.787882 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-1173452c-993d-4b9f-a834-60abcc979d8f tempest-VolumeQuotaClassesTest-1228394397 tempest-VolumeQuotaClassesTest-1228394397-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f81b53e3ac924a6f8bef022378ec9e33', 'request_id': 'req-1173452c-993d-4b9f-a834-60abcc979d8f'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:34.798798 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-1173452c-993d-4b9f-a834-60abcc979d8f tempest-VolumeQuotaClassesTest-1228394397 tempest-VolumeQuotaClassesTest-1228394397-project-admin] The token cache is being invalidate because project f81b53e3ac924a6f8bef022378ec9e33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:47:34.799084 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-1173452c-993d-4b9f-a834-60abcc979d8f tempest-VolumeQuotaClassesTest-1228394397 tempest-VolumeQuotaClassesTest-1228394397-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:34.799752 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-1173452c-993d-4b9f-a834-60abcc979d8f tempest-VolumeQuotaClassesTest-1228394397 tempest-VolumeQuotaClassesTest-1228394397-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:34.801339 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1087/2172] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:47:34 2022] DELETE /identity/v3/projects/f81b53e3ac924a6f8bef022378ec9e33 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:34.954227 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:35.086087 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:35.209262 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:35.428838 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e72dc4-fbba-43bb-98bc-e674e445ed0e tempest-VolumeQuotaClassesTest-1228394397 tempest-VolumeQuotaClassesTest-1228394397-project-admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:35.430689 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-25e72dc4-fbba-43bb-98bc-e674e445ed0e tempest-VolumeQuotaClassesTest-1228394397 tempest-VolumeQuotaClassesTest-1228394397-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:35.499707 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e72dc4-fbba-43bb-98bc-e674e445ed0e tempest-VolumeQuotaClassesTest-1228394397 tempest-VolumeQuotaClassesTest-1228394397-project-admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:35.555774 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f202d5c-a973-4759-ae28-6f8486ebf14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:35.557234 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f202d5c-a973-4759-ae28-6f8486ebf14d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:35.557375 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f202d5c-a973-4759-ae28-6f8486ebf14d admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:35.557648 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f202d5c-a973-4759-ae28-6f8486ebf14d admin admin] PATH_INFO: `/v3/users/8947243c85c34979b6aa821f741d89e8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:35.558853 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f202d5c-a973-4759-ae28-6f8486ebf14d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8947243c85c34979b6aa821f741d89e8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:35.560782 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f202d5c-a973-4759-ae28-6f8486ebf14d admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:35.614835 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:35.758144 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:35.812290 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3f202d5c-a973-4759-ae28-6f8486ebf14d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8947243c85c34979b6aa821f741d89e8'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:35.920535 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3f202d5c-a973-4759-ae28-6f8486ebf14d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8947243c85c34979b6aa821f741d89e8'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:35.928423 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3f202d5c-a973-4759-ae28-6f8486ebf14d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8947243c85c34979b6aa821f741d89e8'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:35.943717 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3f202d5c-a973-4759-ae28-6f8486ebf14d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8947243c85c34979b6aa821f741d89e8'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:35.944662 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3f202d5c-a973-4759-ae28-6f8486ebf14d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8947243c85c34979b6aa821f741d89e8'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:35.951655 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1086/2173] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:47:35 2022] DELETE /identity/v3/users/8947243c85c34979b6aa821f741d89e8 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:35.956789 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2961870b-4f7e-4a0a-80b1-4ca5f2e7dedd None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:35.957098 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2961870b-4f7e-4a0a-80b1-4ca5f2e7dedd None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:35.957379 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2961870b-4f7e-4a0a-80b1-4ca5f2e7dedd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:35.975773 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-2961870b-4f7e-4a0a-80b1-4ca5f2e7dedd None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:35.976991 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-2961870b-4f7e-4a0a-80b1-4ca5f2e7dedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:36.040715 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1088/2174] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:36.051982 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f202d5c-a973-4759-ae28-6f8486ebf14d admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:36.054378 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-3f202d5c-a973-4759-ae28-6f8486ebf14d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:36.131073 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f202d5c-a973-4759-ae28-6f8486ebf14d admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:36.135462 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e469b-9181-4bd7-ba95-552980786930 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:36.136160 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e469b-9181-4bd7-ba95-552980786930 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:36.136685 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e469b-9181-4bd7-ba95-552980786930 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:36.136977 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e469b-9181-4bd7-ba95-552980786930 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:36.149176 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7e469b-9181-4bd7-ba95-552980786930 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:36.150725 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7e469b-9181-4bd7-ba95-552980786930 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:36.162337 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1087/2175] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:47:36.286087 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:36.389034 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:36.506673 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:36.693315 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2961870b-4f7e-4a0a-80b1-4ca5f2e7dedd None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:36.695063 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-2961870b-4f7e-4a0a-80b1-4ca5f2e7dedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:36.709148 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2961870b-4f7e-4a0a-80b1-4ca5f2e7dedd None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:36.716233 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89024df2-eb7e-443a-859a-ed732a0c5f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:36.717154 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89024df2-eb7e-443a-859a-ed732a0c5f2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:36.717439 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89024df2-eb7e-443a-859a-ed732a0c5f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:36.717716 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89024df2-eb7e-443a-859a-ed732a0c5f2d admin admin] PATH_INFO: `/v3/projects/ebf10aef0e65482dbba183d25f564bbe` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:36.720262 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89024df2-eb7e-443a-859a-ed732a0c5f2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebf10aef0e65482dbba183d25f564bbe)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:36.721002 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89024df2-eb7e-443a-859a-ed732a0c5f2d admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:36.777558 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:36.822874 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-89024df2-eb7e-443a-859a-ed732a0c5f2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebf10aef0e65482dbba183d25f564bbe', 'request_id': 'req-89024df2-eb7e-443a-859a-ed732a0c5f2d'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:36.829661 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-89024df2-eb7e-443a-859a-ed732a0c5f2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebf10aef0e65482dbba183d25f564bbe', 'request_id': 'req-89024df2-eb7e-443a-859a-ed732a0c5f2d'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:36.840447 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:36.909046 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-89024df2-eb7e-443a-859a-ed732a0c5f2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebf10aef0e65482dbba183d25f564bbe', 'request_id': 'req-89024df2-eb7e-443a-859a-ed732a0c5f2d'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:36.920523 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-89024df2-eb7e-443a-859a-ed732a0c5f2d admin admin] The token cache is being invalidate because project ebf10aef0e65482dbba183d25f564bbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:47:36.920812 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-89024df2-eb7e-443a-859a-ed732a0c5f2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:36.921461 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-89024df2-eb7e-443a-859a-ed732a0c5f2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:36.923048 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1089/2176] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:47:36 2022] DELETE /identity/v3/projects/ebf10aef0e65482dbba183d25f564bbe => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:36.927078 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e469b-9181-4bd7-ba95-552980786930 service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:36.936972 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e469b-9181-4bd7-ba95-552980786930 service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:36.999644 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472f0ea9-ca50-4536-bb97-406541de9a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:37.000549 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f0ea9-ca50-4536-bb97-406541de9a49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:37.000831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f0ea9-ca50-4536-bb97-406541de9a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:37.001107 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f0ea9-ca50-4536-bb97-406541de9a49 admin admin] PATH_INFO: `/v3/users/8f2eda535fe54da78f37a2b8c88cbec6` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:37.002463 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472f0ea9-ca50-4536-bb97-406541de9a49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f2eda535fe54da78f37a2b8c88cbec6)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:37.004505 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472f0ea9-ca50-4536-bb97-406541de9a49 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:37.073265 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:37.100845 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:37.226385 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-472f0ea9-ca50-4536-bb97-406541de9a49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f2eda535fe54da78f37a2b8c88cbec6'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:37.354306 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-472f0ea9-ca50-4536-bb97-406541de9a49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f2eda535fe54da78f37a2b8c88cbec6'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:37.362083 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-472f0ea9-ca50-4536-bb97-406541de9a49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f2eda535fe54da78f37a2b8c88cbec6'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:37.375956 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-472f0ea9-ca50-4536-bb97-406541de9a49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f2eda535fe54da78f37a2b8c88cbec6'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:37.376685 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-472f0ea9-ca50-4536-bb97-406541de9a49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f2eda535fe54da78f37a2b8c88cbec6'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:37.382765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1088/2177] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:47:36 2022] DELETE /identity/v3/users/8f2eda535fe54da78f37a2b8c88cbec6 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:37.440297 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:37.564688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:37.657768 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:37.946206 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89024df2-eb7e-443a-859a-ed732a0c5f2d admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:37.947991 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-89024df2-eb7e-443a-859a-ed732a0c5f2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:37.961869 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89024df2-eb7e-443a-859a-ed732a0c5f2d admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:38.036240 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea28d55-fc05-44b3-b5e0-5c594d0145d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:38.037123 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea28d55-fc05-44b3-b5e0-5c594d0145d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:38.037409 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea28d55-fc05-44b3-b5e0-5c594d0145d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:38.037688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea28d55-fc05-44b3-b5e0-5c594d0145d0 admin admin] PATH_INFO: `/v3/projects/764985ac006a443095664f8ec2ac4479` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:38.038885 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea28d55-fc05-44b3-b5e0-5c594d0145d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=764985ac006a443095664f8ec2ac4479)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:38.041762 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea28d55-fc05-44b3-b5e0-5c594d0145d0 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:38.164643 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-0ea28d55-fc05-44b3-b5e0-5c594d0145d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '764985ac006a443095664f8ec2ac4479', 'request_id': 'req-0ea28d55-fc05-44b3-b5e0-5c594d0145d0'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:38.171948 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-0ea28d55-fc05-44b3-b5e0-5c594d0145d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '764985ac006a443095664f8ec2ac4479', 'request_id': 'req-0ea28d55-fc05-44b3-b5e0-5c594d0145d0'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:38.186943 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:38.279759 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-0ea28d55-fc05-44b3-b5e0-5c594d0145d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '764985ac006a443095664f8ec2ac4479', 'request_id': 'req-0ea28d55-fc05-44b3-b5e0-5c594d0145d0'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:38.290792 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-0ea28d55-fc05-44b3-b5e0-5c594d0145d0 admin admin] The token cache is being invalidate because project 764985ac006a443095664f8ec2ac4479 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:47:38.292858 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-0ea28d55-fc05-44b3-b5e0-5c594d0145d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:38.292858 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-0ea28d55-fc05-44b3-b5e0-5c594d0145d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:38.293382 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1090/2178] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:47:37 2022] DELETE /identity/v3/projects/764985ac006a443095664f8ec2ac4479 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:38.299565 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd1f9d-90ff-43d7-9e10-f2f64a10aa11 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:38.300680 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd1f9d-90ff-43d7-9e10-f2f64a10aa11 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:38.301516 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd1f9d-90ff-43d7-9e10-f2f64a10aa11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:38.323865 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-01fd1f9d-90ff-43d7-9e10-f2f64a10aa11 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:38.325343 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-01fd1f9d-90ff-43d7-9e10-f2f64a10aa11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:38.389573 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1089/2179] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:38.409460 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf2193-19cc-43fc-9b56-00e2bc2919c0 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:38.409743 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf2193-19cc-43fc-9b56-00e2bc2919c0 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:38.410021 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf2193-19cc-43fc-9b56-00e2bc2919c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:38.428879 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-9adf2193-19cc-43fc-9b56-00e2bc2919c0 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:38.430070 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-9adf2193-19cc-43fc-9b56-00e2bc2919c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:38.438238 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1091/2180] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:38.443210 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd1f9d-90ff-43d7-9e10-f2f64a10aa11 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:38.453885 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd1f9d-90ff-43d7-9e10-f2f64a10aa11 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:38.458371 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80dbdd8-40b4-4dc9-b285-e2af637f0f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:38.459023 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80dbdd8-40b4-4dc9-b285-e2af637f0f4b admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:38.459984 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80dbdd8-40b4-4dc9-b285-e2af637f0f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:38.459984 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80dbdd8-40b4-4dc9-b285-e2af637f0f4b admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:38.460327 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80dbdd8-40b4-4dc9-b285-e2af637f0f4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:47:38.460632 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80dbdd8-40b4-4dc9-b285-e2af637f0f4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:38.462238 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80dbdd8-40b4-4dc9-b285-e2af637f0f4b admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:38.474586 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1090/2181] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 16:47:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:47:38.479273 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adf2193-19cc-43fc-9b56-00e2bc2919c0 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:38.489152 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adf2193-19cc-43fc-9b56-00e2bc2919c0 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:38.493913 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b29862-f36a-4b06-92de-c2a18ed3b1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:38.494582 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b29862-f36a-4b06-92de-c2a18ed3b1b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:38.506243 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b29862-f36a-4b06-92de-c2a18ed3b1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:38.506243 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b29862-f36a-4b06-92de-c2a18ed3b1b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:38.506243 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b29862-f36a-4b06-92de-c2a18ed3b1b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:38.506243 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b29862-f36a-4b06-92de-c2a18ed3b1b5 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:38.635658 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1092/2182] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:47:38 2022] POST /identity/v3/projects => generated 381 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:47:38.640528 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80dbdd8-40b4-4dc9-b285-e2af637f0f4b admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:38.650648 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80dbdd8-40b4-4dc9-b285-e2af637f0f4b admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:38.651323 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:38.707000 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319b2af4-79e6-4120-8087-9df875bffc9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:38.707765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b2af4-79e6-4120-8087-9df875bffc9b admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:38.708062 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b2af4-79e6-4120-8087-9df875bffc9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:38.708381 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b2af4-79e6-4120-8087-9df875bffc9b admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:38.709124 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319b2af4-79e6-4120-8087-9df875bffc9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:38.711191 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319b2af4-79e6-4120-8087-9df875bffc9b admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:38.800364 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1091/2183] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:47:38 2022] POST /identity/v3/users => generated 291 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:47:38.804490 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b29862-f36a-4b06-92de-c2a18ed3b1b5 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:38.815712 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b29862-f36a-4b06-92de-c2a18ed3b1b5 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:38.822202 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822a4829-341e-4c8d-b406-b08d19c160e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:38.822914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a4829-341e-4c8d-b406-b08d19c160e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:38.823198 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a4829-341e-4c8d-b406-b08d19c160e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:38.823568 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a4829-341e-4c8d-b406-b08d19c160e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:38.824835 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822a4829-341e-4c8d-b406-b08d19c160e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:38.826522 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822a4829-341e-4c8d-b406-b08d19c160e9 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:38.838795 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1093/2184] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:47:38.843017 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319b2af4-79e6-4120-8087-9df875bffc9b admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:38.854924 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319b2af4-79e6-4120-8087-9df875bffc9b admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:38.859738 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9474c0-5776-4628-8a2e-d79e3cdd85f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:38.860427 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9474c0-5776-4628-8a2e-d79e3cdd85f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:38.860715 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9474c0-5776-4628-8a2e-d79e3cdd85f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:38.864939 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9474c0-5776-4628-8a2e-d79e3cdd85f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:38.864939 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9474c0-5776-4628-8a2e-d79e3cdd85f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:38.864939 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9474c0-5776-4628-8a2e-d79e3cdd85f5 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:38.873763 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1092/2185] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:47:38.878050 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822a4829-341e-4c8d-b406-b08d19c160e9 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:38.888552 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822a4829-341e-4c8d-b406-b08d19c160e9 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:38.889367 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:38.893518 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0a667b-e2e0-440f-9187-32dbb7c29be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:38.894448 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0a667b-e2e0-440f-9187-32dbb7c29be1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:38.899405 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0a667b-e2e0-440f-9187-32dbb7c29be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:38.899405 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0a667b-e2e0-440f-9187-32dbb7c29be1 admin admin] PATH_INFO: `/v3/projects/e01c6e2b7d3947f3a69b8020c4766dea/users/6c90e6f06c02409085b8ed717e184765/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:38.910598 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0a667b-e2e0-440f-9187-32dbb7c29be1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e01c6e2b7d3947f3a69b8020c4766dea, user_id=6c90e6f06c02409085b8ed717e184765, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:38.913275 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0a667b-e2e0-440f-9187-32dbb7c29be1 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:39.000604 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1094/2186] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:47:38 2022] PUT /identity/v3/projects/e01c6e2b7d3947f3a69b8020c4766dea/users/6c90e6f06c02409085b8ed717e184765/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:39.013077 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2793df-52e3-4570-b4a5-18116f5409ea None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:39.013373 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2793df-52e3-4570-b4a5-18116f5409ea None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:39.013659 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2793df-52e3-4570-b4a5-18116f5409ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:39.044089 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-5d2793df-52e3-4570-b4a5-18116f5409ea None None] MFA Rules not processed for user `6c90e6f06c02409085b8ed717e184765`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:39.045326 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-5d2793df-52e3-4570-b4a5-18116f5409ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:39.060000 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:39.117015 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1093/2187] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:39.121881 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0a667b-e2e0-440f-9187-32dbb7c29be1 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:39.126597 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:39.188492 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0a667b-e2e0-440f-9187-32dbb7c29be1 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:39.193273 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe2c7cc-4830-429f-ba94-b6ed42178c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:39.193976 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe2c7cc-4830-429f-ba94-b6ed42178c22 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:39.194257 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe2c7cc-4830-429f-ba94-b6ed42178c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:39.194533 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe2c7cc-4830-429f-ba94-b6ed42178c22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:39.195258 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe2c7cc-4830-429f-ba94-b6ed42178c22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:39.199799 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe2c7cc-4830-429f-ba94-b6ed42178c22 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:39.305424 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1095/2188] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:47:39 2022] POST /identity/v3/projects => generated 379 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:47:39.307033 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2793df-52e3-4570-b4a5-18116f5409ea None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:39.320603 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2793df-52e3-4570-b4a5-18116f5409ea None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:39.386601 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea880c2-d541-4f23-a32f-07172ef29bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:39.387280 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea880c2-d541-4f23-a32f-07172ef29bf9 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:39.387609 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea880c2-d541-4f23-a32f-07172ef29bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:39.387889 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea880c2-d541-4f23-a32f-07172ef29bf9 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:39.388608 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea880c2-d541-4f23-a32f-07172ef29bf9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:39.390596 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea880c2-d541-4f23-a32f-07172ef29bf9 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:39.526312 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1094/2189] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:47:39 2022] POST /identity/v3/users => generated 296 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:47:39.530313 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe2c7cc-4830-429f-ba94-b6ed42178c22 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:39.540738 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe2c7cc-4830-429f-ba94-b6ed42178c22 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:39.545607 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac7b402-1d24-4ac2-982a-a35a31aa71ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:39.546276 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac7b402-1d24-4ac2-982a-a35a31aa71ec admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:39.546564 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac7b402-1d24-4ac2-982a-a35a31aa71ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:39.546844 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac7b402-1d24-4ac2-982a-a35a31aa71ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:39.547665 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac7b402-1d24-4ac2-982a-a35a31aa71ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:39.549279 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac7b402-1d24-4ac2-982a-a35a31aa71ec admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:39.561033 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1096/2190] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:47:39.565517 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea880c2-d541-4f23-a32f-07172ef29bf9 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:39.577762 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea880c2-d541-4f23-a32f-07172ef29bf9 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:39.582653 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4a4307-889b-49b5-b358-a0b51b072821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:39.583113 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:INSERT=5,keystone:SELECT=88,keystone:DELETE=7 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:39.583775 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a4307-889b-49b5-b358-a0b51b072821 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:39.584052 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a4307-889b-49b5-b358-a0b51b072821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:39.584325 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a4307-889b-49b5-b358-a0b51b072821 admin admin] PATH_INFO: `/v3/projects/5aa63f6203fd40ee90cfdb2ad6f64993/users/eb67c4e12b20418d863ab85e231b875b/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:39.600554 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4a4307-889b-49b5-b358-a0b51b072821 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5aa63f6203fd40ee90cfdb2ad6f64993, user_id=eb67c4e12b20418d863ab85e231b875b, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:39.603363 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4a4307-889b-49b5-b358-a0b51b072821 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:39.692981 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1095/2191] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:47:39 2022] PUT /identity/v3/projects/5aa63f6203fd40ee90cfdb2ad6f64993/users/eb67c4e12b20418d863ab85e231b875b/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:39.700000 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac7b402-1d24-4ac2-982a-a35a31aa71ec admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:39.706836 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:39.710240 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac7b402-1d24-4ac2-982a-a35a31aa71ec admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:39.771678 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b122e5-a801-4690-95ad-f978bee6f92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:39.772490 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b122e5-a801-4690-95ad-f978bee6f92f admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:39.772856 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b122e5-a801-4690-95ad-f978bee6f92f admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:39.773139 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b122e5-a801-4690-95ad-f978bee6f92f admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:39.773882 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b122e5-a801-4690-95ad-f978bee6f92f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:39.776534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b122e5-a801-4690-95ad-f978bee6f92f admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:39.789460 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1097/2192] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:39 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:47:39.793535 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4a4307-889b-49b5-b358-a0b51b072821 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:39.804272 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4a4307-889b-49b5-b358-a0b51b072821 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:39.810027 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4b3ea4-2da0-482a-aecc-8c460f180530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:39.812438 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4b3ea4-2da0-482a-aecc-8c460f180530 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:39.812438 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4b3ea4-2da0-482a-aecc-8c460f180530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:39.812438 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4b3ea4-2da0-482a-aecc-8c460f180530 admin admin] PATH_INFO: `/v3/projects/5aa63f6203fd40ee90cfdb2ad6f64993/users/eb67c4e12b20418d863ab85e231b875b/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:39.813700 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4b3ea4-2da0-482a-aecc-8c460f180530 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5aa63f6203fd40ee90cfdb2ad6f64993, user_id=eb67c4e12b20418d863ab85e231b875b, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:39.816433 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4b3ea4-2da0-482a-aecc-8c460f180530 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:39.828068 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1096/2193] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:47:39 2022] PUT /identity/v3/projects/5aa63f6203fd40ee90cfdb2ad6f64993/users/eb67c4e12b20418d863ab85e231b875b/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:39.840225 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f498f-0f9c-4dee-be46-7655ae755ef9 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:39.840614 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f498f-0f9c-4dee-be46-7655ae755ef9 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:39.840903 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f498f-0f9c-4dee-be46-7655ae755ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:39.867527 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-0d4f498f-0f9c-4dee-be46-7655ae755ef9 None None] MFA Rules not processed for user `eb67c4e12b20418d863ab85e231b875b`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:39.869645 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-0d4f498f-0f9c-4dee-be46-7655ae755ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:39.939375 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:39.943387 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1098/2194] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:39 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:47:39.955341 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4b3ea4-2da0-482a-aecc-8c460f180530 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:39.957111 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-7d4b3ea4-2da0-482a-aecc-8c460f180530 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:39.997796 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:40.021627 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4b3ea4-2da0-482a-aecc-8c460f180530 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:40.026050 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf5baae-b8c7-4a30-8451-863a0186a610 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:40.030431 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5baae-b8c7-4a30-8451-863a0186a610 service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:40.030431 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5baae-b8c7-4a30-8451-863a0186a610 service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:40.030431 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5baae-b8c7-4a30-8451-863a0186a610 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:40.037954 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf5baae-b8c7-4a30-8451-863a0186a610 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:40.039468 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf5baae-b8c7-4a30-8451-863a0186a610 service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:40.050709 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1097/2195] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:39 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:47:40.169771 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4f498f-0f9c-4dee-be46-7655ae755ef9 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:40.171548 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-0d4f498f-0f9c-4dee-be46-7655ae755ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:40.183685 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:40.184155 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4f498f-0f9c-4dee-be46-7655ae755ef9 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:40.187901 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cab9be-10ae-45ec-9cd7-e3145d30954d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:40.191127 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cab9be-10ae-45ec-9cd7-e3145d30954d service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:40.191127 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cab9be-10ae-45ec-9cd7-e3145d30954d service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:40.191127 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cab9be-10ae-45ec-9cd7-e3145d30954d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:40.214357 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:40.254776 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cab9be-10ae-45ec-9cd7-e3145d30954d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:40.256322 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cab9be-10ae-45ec-9cd7-e3145d30954d service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:40.266708 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1099/2196] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:47:40.423664 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:40.835037 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:40.969855 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:41.741054 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:42.148693 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:44.766388 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf5baae-b8c7-4a30-8451-863a0186a610 service cinder] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:44.840197 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf5baae-b8c7-4a30-8451-863a0186a610 service cinder] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:44.845174 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d7e578-9b3d-4ca5-a333-12214048f1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:44.846054 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d7e578-9b3d-4ca5-a333-12214048f1d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:44.846346 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d7e578-9b3d-4ca5-a333-12214048f1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:44.846625 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d7e578-9b3d-4ca5-a333-12214048f1d5 admin admin] PATH_INFO: `/v3/users/6c90e6f06c02409085b8ed717e184765` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:44.847984 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d7e578-9b3d-4ca5-a333-12214048f1d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c90e6f06c02409085b8ed717e184765)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:44.850230 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d7e578-9b3d-4ca5-a333-12214048f1d5 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:45.120695 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-13d7e578-9b3d-4ca5-a333-12214048f1d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c90e6f06c02409085b8ed717e184765'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:45.123130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:45.443337 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-13d7e578-9b3d-4ca5-a333-12214048f1d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c90e6f06c02409085b8ed717e184765'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:45.452019 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-13d7e578-9b3d-4ca5-a333-12214048f1d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c90e6f06c02409085b8ed717e184765'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:45.466941 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-13d7e578-9b3d-4ca5-a333-12214048f1d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c90e6f06c02409085b8ed717e184765'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:45.467756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-13d7e578-9b3d-4ca5-a333-12214048f1d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c90e6f06c02409085b8ed717e184765'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:45.474726 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1098/2197] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:47:44 2022] DELETE /identity/v3/users/6c90e6f06c02409085b8ed717e184765 => generated 0 bytes in 709 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:45.480058 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4928500-96e9-46d3-8a6b-b4d5538a244c None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:45.488370 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4928500-96e9-46d3-8a6b-b4d5538a244c None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:45.488370 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4928500-96e9-46d3-8a6b-b4d5538a244c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:45.499015 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-c4928500-96e9-46d3-8a6b-b4d5538a244c None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:45.500278 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-c4928500-96e9-46d3-8a6b-b4d5538a244c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:45.560521 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1100/2198] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:45.570935 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d7e578-9b3d-4ca5-a333-12214048f1d5 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:45.573524 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-13d7e578-9b3d-4ca5-a333-12214048f1d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:45.639145 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d7e578-9b3d-4ca5-a333-12214048f1d5 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:45.642915 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd4e7ff-ab03-496f-9131-f4282d0dce86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:45.644026 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd4e7ff-ab03-496f-9131-f4282d0dce86 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:45.644026 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd4e7ff-ab03-496f-9131-f4282d0dce86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:45.644229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd4e7ff-ab03-496f-9131-f4282d0dce86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:45.655809 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd4e7ff-ab03-496f-9131-f4282d0dce86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:45.657261 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd4e7ff-ab03-496f-9131-f4282d0dce86 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:45.668478 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1099/2199] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:47:46.148407 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4928500-96e9-46d3-8a6b-b4d5538a244c None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:46.150019 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-c4928500-96e9-46d3-8a6b-b4d5538a244c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:46.164435 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4928500-96e9-46d3-8a6b-b4d5538a244c None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:46.170200 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff7adaa-627a-4d65-b446-fbd1adb8a1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:46.171100 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff7adaa-627a-4d65-b446-fbd1adb8a1fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:46.171381 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff7adaa-627a-4d65-b446-fbd1adb8a1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:46.172034 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff7adaa-627a-4d65-b446-fbd1adb8a1fb admin admin] PATH_INFO: `/v3/projects/e01c6e2b7d3947f3a69b8020c4766dea` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:46.172951 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff7adaa-627a-4d65-b446-fbd1adb8a1fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=e01c6e2b7d3947f3a69b8020c4766dea)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:46.176012 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff7adaa-627a-4d65-b446-fbd1adb8a1fb admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:46.320335 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:46.326048 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-7ff7adaa-627a-4d65-b446-fbd1adb8a1fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e01c6e2b7d3947f3a69b8020c4766dea', 'request_id': 'req-7ff7adaa-627a-4d65-b446-fbd1adb8a1fb'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:46.332826 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-7ff7adaa-627a-4d65-b446-fbd1adb8a1fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e01c6e2b7d3947f3a69b8020c4766dea', 'request_id': 'req-7ff7adaa-627a-4d65-b446-fbd1adb8a1fb'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:46.403814 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-7ff7adaa-627a-4d65-b446-fbd1adb8a1fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e01c6e2b7d3947f3a69b8020c4766dea', 'request_id': 'req-7ff7adaa-627a-4d65-b446-fbd1adb8a1fb'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:46.416979 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-7ff7adaa-627a-4d65-b446-fbd1adb8a1fb admin admin] The token cache is being invalidate because project e01c6e2b7d3947f3a69b8020c4766dea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:47:46.417271 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-7ff7adaa-627a-4d65-b446-fbd1adb8a1fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:46.417954 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-7ff7adaa-627a-4d65-b446-fbd1adb8a1fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:46.419569 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1101/2200] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:47:46 2022] DELETE /identity/v3/projects/e01c6e2b7d3947f3a69b8020c4766dea => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:46.423559 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd4e7ff-ab03-496f-9131-f4282d0dce86 service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:46.433933 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd4e7ff-ab03-496f-9131-f4282d0dce86 service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:46.491258 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec78f84-cfe3-48f1-8829-d5a68fda2bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:46.492165 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec78f84-cfe3-48f1-8829-d5a68fda2bcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:46.492448 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec78f84-cfe3-48f1-8829-d5a68fda2bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:46.492722 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec78f84-cfe3-48f1-8829-d5a68fda2bcc admin admin] PATH_INFO: `/v3/users/eb67c4e12b20418d863ab85e231b875b` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:46.493899 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec78f84-cfe3-48f1-8829-d5a68fda2bcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb67c4e12b20418d863ab85e231b875b)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:46.495899 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec78f84-cfe3-48f1-8829-d5a68fda2bcc admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:46.733921 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-2ec78f84-cfe3-48f1-8829-d5a68fda2bcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb67c4e12b20418d863ab85e231b875b'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:46.823768 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-2ec78f84-cfe3-48f1-8829-d5a68fda2bcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb67c4e12b20418d863ab85e231b875b'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:46.831876 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-2ec78f84-cfe3-48f1-8829-d5a68fda2bcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb67c4e12b20418d863ab85e231b875b'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:46.840300 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:46.845535 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-2ec78f84-cfe3-48f1-8829-d5a68fda2bcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb67c4e12b20418d863ab85e231b875b'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:46.846258 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-2ec78f84-cfe3-48f1-8829-d5a68fda2bcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb67c4e12b20418d863ab85e231b875b'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:46.852247 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1100/2201] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:47:46 2022] DELETE /identity/v3/users/eb67c4e12b20418d863ab85e231b875b => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:47.300437 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff7adaa-627a-4d65-b446-fbd1adb8a1fb admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:47.302222 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-7ff7adaa-627a-4d65-b446-fbd1adb8a1fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:47.319057 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff7adaa-627a-4d65-b446-fbd1adb8a1fb admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:47.384154 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d32a2e4-484b-4cfb-b680-cb8ecc8ded41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:47.385087 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d32a2e4-484b-4cfb-b680-cb8ecc8ded41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:47.385440 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d32a2e4-484b-4cfb-b680-cb8ecc8ded41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:47.385722 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d32a2e4-484b-4cfb-b680-cb8ecc8ded41 admin admin] PATH_INFO: `/v3/projects/5aa63f6203fd40ee90cfdb2ad6f64993` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:47.386921 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d32a2e4-484b-4cfb-b680-cb8ecc8ded41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5aa63f6203fd40ee90cfdb2ad6f64993)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:47.388864 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d32a2e4-484b-4cfb-b680-cb8ecc8ded41 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:47.534287 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-8d32a2e4-484b-4cfb-b680-cb8ecc8ded41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5aa63f6203fd40ee90cfdb2ad6f64993', 'request_id': 'req-8d32a2e4-484b-4cfb-b680-cb8ecc8ded41'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:47.540443 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-8d32a2e4-484b-4cfb-b680-cb8ecc8ded41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5aa63f6203fd40ee90cfdb2ad6f64993', 'request_id': 'req-8d32a2e4-484b-4cfb-b680-cb8ecc8ded41'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:47.634468 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-8d32a2e4-484b-4cfb-b680-cb8ecc8ded41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5aa63f6203fd40ee90cfdb2ad6f64993', 'request_id': 'req-8d32a2e4-484b-4cfb-b680-cb8ecc8ded41'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:47.649161 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-8d32a2e4-484b-4cfb-b680-cb8ecc8ded41 admin admin] The token cache is being invalidate because project 5aa63f6203fd40ee90cfdb2ad6f64993 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:47:47.649453 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-8d32a2e4-484b-4cfb-b680-cb8ecc8ded41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:47.650195 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-8d32a2e4-484b-4cfb-b680-cb8ecc8ded41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:47.652190 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1102/2202] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:47:47 2022] DELETE /identity/v3/projects/5aa63f6203fd40ee90cfdb2ad6f64993 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:47.657069 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d25b97-4fa3-4b70-8959-945d3c95da25 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:47.657532 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d25b97-4fa3-4b70-8959-945d3c95da25 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:47.657638 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d25b97-4fa3-4b70-8959-945d3c95da25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:47.675863 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-43d25b97-4fa3-4b70-8959-945d3c95da25 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:47.677061 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-43d25b97-4fa3-4b70-8959-945d3c95da25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:47.745485 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1101/2203] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:47.749347 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0ac9e4-03f0-40d5-b5d1-cf063dea0527 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:47.749644 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0ac9e4-03f0-40d5-b5d1-cf063dea0527 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:47.749928 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0ac9e4-03f0-40d5-b5d1-cf063dea0527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:47.770889 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-da0ac9e4-03f0-40d5-b5d1-cf063dea0527 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:47.771880 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-da0ac9e4-03f0-40d5-b5d1-cf063dea0527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:47.785562 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1103/2204] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:47.788842 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d25b97-4fa3-4b70-8959-945d3c95da25 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:47.799479 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d25b97-4fa3-4b70-8959-945d3c95da25 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:47.804220 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b398c49-fa55-46f1-8f3e-6bc68f8b5863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:47.804877 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b398c49-fa55-46f1-8f3e-6bc68f8b5863 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:47.805156 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b398c49-fa55-46f1-8f3e-6bc68f8b5863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:47.805431 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b398c49-fa55-46f1-8f3e-6bc68f8b5863 admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:47.806148 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b398c49-fa55-46f1-8f3e-6bc68f8b5863 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:47:47.806487 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b398c49-fa55-46f1-8f3e-6bc68f8b5863 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:47.808206 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b398c49-fa55-46f1-8f3e-6bc68f8b5863 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:47.825292 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1102/2205] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 16:47:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:47:47.826347 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0ac9e4-03f0-40d5-b5d1-cf063dea0527 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:47.836289 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0ac9e4-03f0-40d5-b5d1-cf063dea0527 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:47.842962 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a0748-6a15-4133-9363-13de54c25f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:47.842962 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a0748-6a15-4133-9363-13de54c25f5f admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:47.843419 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a0748-6a15-4133-9363-13de54c25f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:47.843672 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a0748-6a15-4133-9363-13de54c25f5f admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:47.844303 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a0748-6a15-4133-9363-13de54c25f5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:47.846607 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a0748-6a15-4133-9363-13de54c25f5f admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:47.963708 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1104/2206] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:47:47 2022] POST /identity/v3/projects => generated 367 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:47:47.967190 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b398c49-fa55-46f1-8f3e-6bc68f8b5863 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:47.977505 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b398c49-fa55-46f1-8f3e-6bc68f8b5863 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:48.036532 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b23abdd-b9d2-48e9-bb81-b04e2c78e641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:48.037126 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23abdd-b9d2-48e9-bb81-b04e2c78e641 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:48.037412 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23abdd-b9d2-48e9-bb81-b04e2c78e641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:48.037752 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23abdd-b9d2-48e9-bb81-b04e2c78e641 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:48.038465 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b23abdd-b9d2-48e9-bb81-b04e2c78e641 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:48.040434 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b23abdd-b9d2-48e9-bb81-b04e2c78e641 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:48.115423 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1103/2207] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:47:47 2022] POST /identity/v3/users => generated 284 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:47:48.119863 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a0748-6a15-4133-9363-13de54c25f5f admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:48.131710 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a0748-6a15-4133-9363-13de54c25f5f admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:48.137238 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eb7f69-8e46-4aaf-b6be-e3c4e97e075a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:48.137919 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb7f69-8e46-4aaf-b6be-e3c4e97e075a admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:48.138202 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb7f69-8e46-4aaf-b6be-e3c4e97e075a admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:48.138477 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb7f69-8e46-4aaf-b6be-e3c4e97e075a admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:48.139230 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eb7f69-8e46-4aaf-b6be-e3c4e97e075a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:48.140878 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eb7f69-8e46-4aaf-b6be-e3c4e97e075a admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:48.152675 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1105/2208] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:47:48.156748 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b23abdd-b9d2-48e9-bb81-b04e2c78e641 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:48.168113 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b23abdd-b9d2-48e9-bb81-b04e2c78e641 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:48.172865 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea9115a-8b87-4742-ae53-67e42ba41e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:48.177204 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9115a-8b87-4742-ae53-67e42ba41e20 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:48.177204 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9115a-8b87-4742-ae53-67e42ba41e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:48.177204 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9115a-8b87-4742-ae53-67e42ba41e20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:48.177204 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea9115a-8b87-4742-ae53-67e42ba41e20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:48.177204 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea9115a-8b87-4742-ae53-67e42ba41e20 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:48.186780 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1104/2209] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:47:48.190954 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eb7f69-8e46-4aaf-b6be-e3c4e97e075a admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:48.205179 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eb7f69-8e46-4aaf-b6be-e3c4e97e075a admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:48.209936 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345df7ac-3e81-4d38-8c77-6e9e7f363cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:48.214637 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345df7ac-3e81-4d38-8c77-6e9e7f363cf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:48.214637 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345df7ac-3e81-4d38-8c77-6e9e7f363cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:48.214637 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345df7ac-3e81-4d38-8c77-6e9e7f363cf8 admin admin] PATH_INFO: `/v3/projects/4418c930433348f7846281d0a027f717/users/a624cdc5dbf247bea5d8049bc708250a/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:48.227864 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345df7ac-3e81-4d38-8c77-6e9e7f363cf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4418c930433348f7846281d0a027f717, user_id=a624cdc5dbf247bea5d8049bc708250a, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:48.230736 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345df7ac-3e81-4d38-8c77-6e9e7f363cf8 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:48.328133 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:48.332945 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1106/2210] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:47:48 2022] PUT /identity/v3/projects/4418c930433348f7846281d0a027f717/users/a624cdc5dbf247bea5d8049bc708250a/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:48.356232 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd200151-457f-4150-9190-eb99d7354f65 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:48.356615 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd200151-457f-4150-9190-eb99d7354f65 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:48.356966 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd200151-457f-4150-9190-eb99d7354f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:48.395648 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-fd200151-457f-4150-9190-eb99d7354f65 None None] MFA Rules not processed for user `a624cdc5dbf247bea5d8049bc708250a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:48.397570 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-fd200151-457f-4150-9190-eb99d7354f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:48.450245 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:48.473740 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1105/2211] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:48 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:48.478336 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345df7ac-3e81-4d38-8c77-6e9e7f363cf8 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:48.555847 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345df7ac-3e81-4d38-8c77-6e9e7f363cf8 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:48.561212 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8955867-b0b1-4645-98a5-4046f130244c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:48.561876 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8955867-b0b1-4645-98a5-4046f130244c admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:48.562161 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8955867-b0b1-4645-98a5-4046f130244c admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:48.562437 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8955867-b0b1-4645-98a5-4046f130244c admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:48.563138 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8955867-b0b1-4645-98a5-4046f130244c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:48.565133 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8955867-b0b1-4645-98a5-4046f130244c admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:48.664665 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1107/2212] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:47:48 2022] POST /identity/v3/projects => generated 365 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:47:48.669274 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd200151-457f-4150-9190-eb99d7354f65 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:48.680519 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd200151-457f-4150-9190-eb99d7354f65 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:48.737820 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d21ab-8c79-4e7c-9196-f2d4e70079a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:48.738507 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d21ab-8c79-4e7c-9196-f2d4e70079a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:48.738788 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d21ab-8c79-4e7c-9196-f2d4e70079a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:48.739062 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d21ab-8c79-4e7c-9196-f2d4e70079a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:48.739799 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110d21ab-8c79-4e7c-9196-f2d4e70079a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:48.741769 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110d21ab-8c79-4e7c-9196-f2d4e70079a7 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:48.815355 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1106/2213] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:47:48 2022] POST /identity/v3/users => generated 289 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:47:48.819342 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8955867-b0b1-4645-98a5-4046f130244c admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:48.832997 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8955867-b0b1-4645-98a5-4046f130244c admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:48.836001 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca1721a-34d5-43c9-9d45-e841adb39805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:48.836672 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca1721a-34d5-43c9-9d45-e841adb39805 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:48.837008 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca1721a-34d5-43c9-9d45-e841adb39805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:48.837227 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca1721a-34d5-43c9-9d45-e841adb39805 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:48.838592 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca1721a-34d5-43c9-9d45-e841adb39805 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:48.840200 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca1721a-34d5-43c9-9d45-e841adb39805 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:48.852543 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1108/2214] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:47:48.857003 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d21ab-8c79-4e7c-9196-f2d4e70079a7 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:48.867691 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d21ab-8c79-4e7c-9196-f2d4e70079a7 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:48.872630 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c3f94-d731-4d34-89ab-a60eedc2556b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:48.873866 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c3f94-d731-4d34-89ab-a60eedc2556b admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:48.873866 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c3f94-d731-4d34-89ab-a60eedc2556b admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:48.874102 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c3f94-d731-4d34-89ab-a60eedc2556b admin admin] PATH_INFO: `/v3/projects/d73e1d57d4a04a1396448a0d22c96066/users/47795f82d672404485e450f41a6318f8/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:48.890151 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829c3f94-d731-4d34-89ab-a60eedc2556b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d73e1d57d4a04a1396448a0d22c96066, user_id=47795f82d672404485e450f41a6318f8, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:48.892890 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829c3f94-d731-4d34-89ab-a60eedc2556b admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:48.976025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1107/2215] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:47:48 2022] PUT /identity/v3/projects/d73e1d57d4a04a1396448a0d22c96066/users/47795f82d672404485e450f41a6318f8/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:48.980244 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca1721a-34d5-43c9-9d45-e841adb39805 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:48.990231 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca1721a-34d5-43c9-9d45-e841adb39805 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:49.047220 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee59c42a-f670-42b6-bb43-5bc14454dfbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:49.048215 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee59c42a-f670-42b6-bb43-5bc14454dfbd admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:49.048499 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee59c42a-f670-42b6-bb43-5bc14454dfbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:49.048774 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee59c42a-f670-42b6-bb43-5bc14454dfbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:49.049493 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee59c42a-f670-42b6-bb43-5bc14454dfbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:49.051079 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee59c42a-f670-42b6-bb43-5bc14454dfbd admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:49.061610 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1109/2216] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:48 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:47:49.065858 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c3f94-d731-4d34-89ab-a60eedc2556b admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:49.077212 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c3f94-d731-4d34-89ab-a60eedc2556b admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:49.081840 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd6a6c1-6d0c-4189-9a03-cfcd3250039a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:49.082715 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd6a6c1-6d0c-4189-9a03-cfcd3250039a admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:49.082994 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd6a6c1-6d0c-4189-9a03-cfcd3250039a admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:49.083269 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd6a6c1-6d0c-4189-9a03-cfcd3250039a admin admin] PATH_INFO: `/v3/projects/d73e1d57d4a04a1396448a0d22c96066/users/47795f82d672404485e450f41a6318f8/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:49.085275 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd6a6c1-6d0c-4189-9a03-cfcd3250039a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d73e1d57d4a04a1396448a0d22c96066, user_id=47795f82d672404485e450f41a6318f8, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:49.090122 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd6a6c1-6d0c-4189-9a03-cfcd3250039a admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:49.100850 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1108/2217] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:47:49 2022] PUT /identity/v3/projects/d73e1d57d4a04a1396448a0d22c96066/users/47795f82d672404485e450f41a6318f8/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:49.110936 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df3178-368c-444b-b3e8-a25b6994448c None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:49.111228 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df3178-368c-444b-b3e8-a25b6994448c None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:49.111545 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df3178-368c-444b-b3e8-a25b6994448c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:49.139351 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-16df3178-368c-444b-b3e8-a25b6994448c None None] MFA Rules not processed for user `47795f82d672404485e450f41a6318f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:49.151368 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-16df3178-368c-444b-b3e8-a25b6994448c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:49.262671 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:49.262671 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1110/2218] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:49 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:49.263148 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd6a6c1-6d0c-4189-9a03-cfcd3250039a admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:49.263148 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-fcd6a6c1-6d0c-4189-9a03-cfcd3250039a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:49.284360 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd6a6c1-6d0c-4189-9a03-cfcd3250039a admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:49.288047 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832aca7c-8c47-4e4f-b3f4-3873571a60c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:49.288697 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832aca7c-8c47-4e4f-b3f4-3873571a60c7 service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:49.288987 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832aca7c-8c47-4e4f-b3f4-3873571a60c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:49.289263 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832aca7c-8c47-4e4f-b3f4-3873571a60c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:49.298717 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832aca7c-8c47-4e4f-b3f4-3873571a60c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:49.300197 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832aca7c-8c47-4e4f-b3f4-3873571a60c7 service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:49.311063 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1109/2219] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:49 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:47:49.573283 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16df3178-368c-444b-b3e8-a25b6994448c None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:49.583140 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16df3178-368c-444b-b3e8-a25b6994448c None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:49.586891 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e40214-7f43-49c3-93e8-1247ae61e909 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:49.588546 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e40214-7f43-49c3-93e8-1247ae61e909 service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:49.588699 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e40214-7f43-49c3-93e8-1247ae61e909 service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:49.588912 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e40214-7f43-49c3-93e8-1247ae61e909 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:49.646853 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e40214-7f43-49c3-93e8-1247ae61e909 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:49.648386 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e40214-7f43-49c3-93e8-1247ae61e909 service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:49.658396 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1111/2220] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:49 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:47:56.225958 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832aca7c-8c47-4e4f-b3f4-3873571a60c7 service cinder] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:56.227466 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-832aca7c-8c47-4e4f-b3f4-3873571a60c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:56.278017 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:56.300540 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832aca7c-8c47-4e4f-b3f4-3873571a60c7 service cinder] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:56.305501 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d97c0-eab3-4a86-880a-7a33d855ed21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:56.306384 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d97c0-eab3-4a86-880a-7a33d855ed21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:56.306730 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d97c0-eab3-4a86-880a-7a33d855ed21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:56.307674 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d97c0-eab3-4a86-880a-7a33d855ed21 admin admin] PATH_INFO: `/v3/users/7389e41d736c4283a8ef8ee074a512e7` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:56.308908 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61d97c0-eab3-4a86-880a-7a33d855ed21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7389e41d736c4283a8ef8ee074a512e7)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:56.310812 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61d97c0-eab3-4a86-880a-7a33d855ed21 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:56.659064 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b61d97c0-eab3-4a86-880a-7a33d855ed21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7389e41d736c4283a8ef8ee074a512e7'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:56.744686 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b61d97c0-eab3-4a86-880a-7a33d855ed21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7389e41d736c4283a8ef8ee074a512e7'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:56.755762 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b61d97c0-eab3-4a86-880a-7a33d855ed21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7389e41d736c4283a8ef8ee074a512e7'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:56.773500 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b61d97c0-eab3-4a86-880a-7a33d855ed21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7389e41d736c4283a8ef8ee074a512e7'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:56.774382 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b61d97c0-eab3-4a86-880a-7a33d855ed21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7389e41d736c4283a8ef8ee074a512e7'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:56.782223 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e40214-7f43-49c3-93e8-1247ae61e909 service cinder] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:56.792231 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1110/2221] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:47:56 2022] DELETE /identity/v3/users/7389e41d736c4283a8ef8ee074a512e7 => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:56.797624 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338f0dd-c027-449c-9edb-cc9e37e20062 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:56.798105 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338f0dd-c027-449c-9edb-cc9e37e20062 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:56.798630 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338f0dd-c027-449c-9edb-cc9e37e20062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:56.814541 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e40214-7f43-49c3-93e8-1247ae61e909 service cinder] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:56.822163 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-6338f0dd-c027-449c-9edb-cc9e37e20062 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:56.823595 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-6338f0dd-c027-449c-9edb-cc9e37e20062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:56.900132 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1111/2222] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:47:56.910649 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6338f0dd-c027-449c-9edb-cc9e37e20062 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:56.912088 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-6338f0dd-c027-449c-9edb-cc9e37e20062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:56.933127 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec9125-0db7-490e-9f5f-b435b4b0dc6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:56.934056 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec9125-0db7-490e-9f5f-b435b4b0dc6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:56.934606 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec9125-0db7-490e-9f5f-b435b4b0dc6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:56.934741 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec9125-0db7-490e-9f5f-b435b4b0dc6f admin admin] PATH_INFO: `/v3/users/a624cdc5dbf247bea5d8049bc708250a` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:56.936053 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ec9125-0db7-490e-9f5f-b435b4b0dc6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a624cdc5dbf247bea5d8049bc708250a)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:56.938346 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ec9125-0db7-490e-9f5f-b435b4b0dc6f admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:56.989835 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6338f0dd-c027-449c-9edb-cc9e37e20062 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:56.993543 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4a9bb6-a9c3-4bd8-9dbd-771b6b59a2fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:56.994119 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a9bb6-a9c3-4bd8-9dbd-771b6b59a2fa service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:56.994406 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a9bb6-a9c3-4bd8-9dbd-771b6b59a2fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:56.994901 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a9bb6-a9c3-4bd8-9dbd-771b6b59a2fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:57.005335 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4a9bb6-a9c3-4bd8-9dbd-771b6b59a2fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:57.006780 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4a9bb6-a9c3-4bd8-9dbd-771b6b59a2fa service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:57.020216 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1112/2223] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:47:57.021652 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4a9bb6-a9c3-4bd8-9dbd-771b6b59a2fa service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:57.023099 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-5d4a9bb6-a9c3-4bd8-9dbd-771b6b59a2fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:57.031930 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:57.038320 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4a9bb6-a9c3-4bd8-9dbd-771b6b59a2fa service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:57.044945 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b095f7-18cd-46df-bde0-3c7f1eb78749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:57.045844 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b095f7-18cd-46df-bde0-3c7f1eb78749 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:57.046123 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b095f7-18cd-46df-bde0-3c7f1eb78749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:57.046391 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b095f7-18cd-46df-bde0-3c7f1eb78749 admin admin] PATH_INFO: `/v3/users/f6126e9b566843beb1bc1e813a5f339f` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:57.047661 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b095f7-18cd-46df-bde0-3c7f1eb78749 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6126e9b566843beb1bc1e813a5f339f)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:57.049579 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b095f7-18cd-46df-bde0-3c7f1eb78749 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:57.138964 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-22ec9125-0db7-490e-9f5f-b435b4b0dc6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a624cdc5dbf247bea5d8049bc708250a'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:57.231046 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-22ec9125-0db7-490e-9f5f-b435b4b0dc6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a624cdc5dbf247bea5d8049bc708250a'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:57.231046 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-22ec9125-0db7-490e-9f5f-b435b4b0dc6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a624cdc5dbf247bea5d8049bc708250a'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:57.232203 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-22ec9125-0db7-490e-9f5f-b435b4b0dc6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a624cdc5dbf247bea5d8049bc708250a'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:57.248454 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-22ec9125-0db7-490e-9f5f-b435b4b0dc6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a624cdc5dbf247bea5d8049bc708250a'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:57.267282 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1112/2224] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:47:56 2022] DELETE /identity/v3/users/a624cdc5dbf247bea5d8049bc708250a => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:47:57.271397 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4b51ed-bb84-43e2-8728-24b5e80a0aea None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:57.277094 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4b51ed-bb84-43e2-8728-24b5e80a0aea None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:57.277094 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4b51ed-bb84-43e2-8728-24b5e80a0aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:57.297435 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-cb4b51ed-bb84-43e2-8728-24b5e80a0aea None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:57.297435 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-cb4b51ed-bb84-43e2-8728-24b5e80a0aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:57.311932 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:57.355588 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-c9b095f7-18cd-46df-bde0-3c7f1eb78749 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6126e9b566843beb1bc1e813a5f339f'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:57.403249 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1113/2225] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:47:57.415300 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4b51ed-bb84-43e2-8728-24b5e80a0aea None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:57.425792 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-c9b095f7-18cd-46df-bde0-3c7f1eb78749 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6126e9b566843beb1bc1e813a5f339f'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:57.426274 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-cb4b51ed-bb84-43e2-8728-24b5e80a0aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:57.428674 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-c9b095f7-18cd-46df-bde0-3c7f1eb78749 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6126e9b566843beb1bc1e813a5f339f'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:57.454397 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-c9b095f7-18cd-46df-bde0-3c7f1eb78749 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6126e9b566843beb1bc1e813a5f339f'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:57.455393 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-c9b095f7-18cd-46df-bde0-3c7f1eb78749 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6126e9b566843beb1bc1e813a5f339f'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:57.463752 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1113/2226] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:47:57 2022] DELETE /identity/v3/users/f6126e9b566843beb1bc1e813a5f339f => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:47:57.469078 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946cd28-db62-4487-a471-34562237b5c6 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:57.469457 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946cd28-db62-4487-a471-34562237b5c6 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:57.469806 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946cd28-db62-4487-a471-34562237b5c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:57.481856 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4b51ed-bb84-43e2-8728-24b5e80a0aea None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:57.488284 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-0946cd28-db62-4487-a471-34562237b5c6 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:57.489477 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-0946cd28-db62-4487-a471-34562237b5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:57.533528 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026543f3-eff9-48fa-b5c3-859d8c68e92e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:57.534233 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026543f3-eff9-48fa-b5c3-859d8c68e92e service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:57.534520 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026543f3-eff9-48fa-b5c3-859d8c68e92e service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:57.534837 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026543f3-eff9-48fa-b5c3-859d8c68e92e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:57.536665 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-026543f3-eff9-48fa-b5c3-859d8c68e92e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:57.561964 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1114/2227] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:47:57.561964 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946cd28-db62-4487-a471-34562237b5c6 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:57.561964 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-0946cd28-db62-4487-a471-34562237b5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:57.581442 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946cd28-db62-4487-a471-34562237b5c6 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:57.588208 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da2e61-d964-4bf1-9b7e-2e7d349c5a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:57.588208 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da2e61-d964-4bf1-9b7e-2e7d349c5a3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:57.588208 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da2e61-d964-4bf1-9b7e-2e7d349c5a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:57.589003 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da2e61-d964-4bf1-9b7e-2e7d349c5a3c admin admin] PATH_INFO: `/v3/projects/2866a5dbf6a64a47839347deb8641c22` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:57.589491 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56da2e61-d964-4bf1-9b7e-2e7d349c5a3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2866a5dbf6a64a47839347deb8641c22)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:57.591405 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56da2e61-d964-4bf1-9b7e-2e7d349c5a3c admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:57.625553 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026543f3-eff9-48fa-b5c3-859d8c68e92e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:57.627338 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026543f3-eff9-48fa-b5c3-859d8c68e92e service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:57.639012 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1114/2228] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:47:57.640111 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026543f3-eff9-48fa-b5c3-859d8c68e92e service neutron] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:57.642611 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-026543f3-eff9-48fa-b5c3-859d8c68e92e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:57.656248 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026543f3-eff9-48fa-b5c3-859d8c68e92e service neutron] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:57.660332 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9132b-9797-4c42-8b53-2b55f7bfd9fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:57.661189 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9132b-9797-4c42-8b53-2b55f7bfd9fe service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:57.661542 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9132b-9797-4c42-8b53-2b55f7bfd9fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:57.662029 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9132b-9797-4c42-8b53-2b55f7bfd9fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:57.673159 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9132b-9797-4c42-8b53-2b55f7bfd9fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:57.674721 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9132b-9797-4c42-8b53-2b55f7bfd9fe service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:57.686115 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1115/2229] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:47:57.718101 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-56da2e61-d964-4bf1-9b7e-2e7d349c5a3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2866a5dbf6a64a47839347deb8641c22', 'request_id': 'req-56da2e61-d964-4bf1-9b7e-2e7d349c5a3c'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:57.724476 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-56da2e61-d964-4bf1-9b7e-2e7d349c5a3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2866a5dbf6a64a47839347deb8641c22', 'request_id': 'req-56da2e61-d964-4bf1-9b7e-2e7d349c5a3c'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:57.731623 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-56da2e61-d964-4bf1-9b7e-2e7d349c5a3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2866a5dbf6a64a47839347deb8641c22', 'request_id': 'req-56da2e61-d964-4bf1-9b7e-2e7d349c5a3c'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:57.796632 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:57.801778 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-56da2e61-d964-4bf1-9b7e-2e7d349c5a3c admin admin] The token cache is being invalidate because project 2866a5dbf6a64a47839347deb8641c22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:47:57.802074 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-56da2e61-d964-4bf1-9b7e-2e7d349c5a3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:57.802732 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-56da2e61-d964-4bf1-9b7e-2e7d349c5a3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:57.804351 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1115/2230] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:47:57 2022] DELETE /identity/v3/projects/2866a5dbf6a64a47839347deb8641c22 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:57.809332 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9132b-9797-4c42-8b53-2b55f7bfd9fe service neutron] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:57.819745 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9132b-9797-4c42-8b53-2b55f7bfd9fe service neutron] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:57.823988 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:57.888813 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a20eff7-1403-4c5a-91ef-7d843e41032a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:57.889649 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a20eff7-1403-4c5a-91ef-7d843e41032a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:57.889936 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a20eff7-1403-4c5a-91ef-7d843e41032a admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:57.890213 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a20eff7-1403-4c5a-91ef-7d843e41032a admin admin] PATH_INFO: `/v3/users/a82b452d649e46a58971bd35bc006cd0` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:57.891447 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a20eff7-1403-4c5a-91ef-7d843e41032a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a82b452d649e46a58971bd35bc006cd0)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:57.893406 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a20eff7-1403-4c5a-91ef-7d843e41032a admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:58.121488 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-7a20eff7-1403-4c5a-91ef-7d843e41032a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a82b452d649e46a58971bd35bc006cd0'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:58.224725 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-7a20eff7-1403-4c5a-91ef-7d843e41032a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a82b452d649e46a58971bd35bc006cd0'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:58.225658 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-7a20eff7-1403-4c5a-91ef-7d843e41032a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a82b452d649e46a58971bd35bc006cd0'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:58.226369 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-7a20eff7-1403-4c5a-91ef-7d843e41032a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a82b452d649e46a58971bd35bc006cd0'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:58.232859 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da2e61-d964-4bf1-9b7e-2e7d349c5a3c admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:58.235981 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-56da2e61-d964-4bf1-9b7e-2e7d349c5a3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:58.256004 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da2e61-d964-4bf1-9b7e-2e7d349c5a3c admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:58.263797 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8d4bc7-04fc-46fd-bb83-8e918abcc523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:58.265019 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-7a20eff7-1403-4c5a-91ef-7d843e41032a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a82b452d649e46a58971bd35bc006cd0'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:58.266982 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8d4bc7-04fc-46fd-bb83-8e918abcc523 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:58.267265 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8d4bc7-04fc-46fd-bb83-8e918abcc523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:58.267596 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8d4bc7-04fc-46fd-bb83-8e918abcc523 admin admin] PATH_INFO: `/v3/projects/4418c930433348f7846281d0a027f717` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:58.269624 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8d4bc7-04fc-46fd-bb83-8e918abcc523 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4418c930433348f7846281d0a027f717)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:58.273161 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8d4bc7-04fc-46fd-bb83-8e918abcc523 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:58.319617 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1116/2231] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:47:57 2022] DELETE /identity/v3/users/a82b452d649e46a58971bd35bc006cd0 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:47:58.319617 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a20eff7-1403-4c5a-91ef-7d843e41032a admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:58.319617 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-7a20eff7-1403-4c5a-91ef-7d843e41032a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:58.423583 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-6e8d4bc7-04fc-46fd-bb83-8e918abcc523 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4418c930433348f7846281d0a027f717', 'request_id': 'req-6e8d4bc7-04fc-46fd-bb83-8e918abcc523'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:58.433159 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-6e8d4bc7-04fc-46fd-bb83-8e918abcc523 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4418c930433348f7846281d0a027f717', 'request_id': 'req-6e8d4bc7-04fc-46fd-bb83-8e918abcc523'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:58.441643 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-6e8d4bc7-04fc-46fd-bb83-8e918abcc523 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4418c930433348f7846281d0a027f717', 'request_id': 'req-6e8d4bc7-04fc-46fd-bb83-8e918abcc523'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:58.454030 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a20eff7-1403-4c5a-91ef-7d843e41032a admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:58.500019 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:58.537671 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-6e8d4bc7-04fc-46fd-bb83-8e918abcc523 admin admin] The token cache is being invalidate because project 4418c930433348f7846281d0a027f717 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:47:58.537974 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-6e8d4bc7-04fc-46fd-bb83-8e918abcc523 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:58.538664 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-6e8d4bc7-04fc-46fd-bb83-8e918abcc523 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:58.540622 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1116/2232] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:47:58 2022] DELETE /identity/v3/projects/4418c930433348f7846281d0a027f717 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:47:58.544690 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8d4bc7-04fc-46fd-bb83-8e918abcc523 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:58.545124 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1070025b-2755-47b9-8c54-e97e940c2377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:58.546611 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1070025b-2755-47b9-8c54-e97e940c2377 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:58.547063 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1070025b-2755-47b9-8c54-e97e940c2377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:58.547438 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1070025b-2755-47b9-8c54-e97e940c2377 admin admin] PATH_INFO: `/v3/projects/ceef800c47564c23a140b7f0e2382a02` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:58.548877 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-6e8d4bc7-04fc-46fd-bb83-8e918abcc523 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:58.549006 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1070025b-2755-47b9-8c54-e97e940c2377 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ceef800c47564c23a140b7f0e2382a02)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:58.550961 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1070025b-2755-47b9-8c54-e97e940c2377 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:58.565564 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8d4bc7-04fc-46fd-bb83-8e918abcc523 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:58.573026 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9678043d-1623-4172-be34-50c7dab5d067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:58.574151 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9678043d-1623-4172-be34-50c7dab5d067 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:58.574523 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9678043d-1623-4172-be34-50c7dab5d067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:58.574864 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9678043d-1623-4172-be34-50c7dab5d067 admin admin] PATH_INFO: `/v3/users/47795f82d672404485e450f41a6318f8` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:58.576974 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9678043d-1623-4172-be34-50c7dab5d067 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47795f82d672404485e450f41a6318f8)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:58.579538 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9678043d-1623-4172-be34-50c7dab5d067 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:58.719741 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-1070025b-2755-47b9-8c54-e97e940c2377 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ceef800c47564c23a140b7f0e2382a02', 'request_id': 'req-1070025b-2755-47b9-8c54-e97e940c2377'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:58.726162 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-1070025b-2755-47b9-8c54-e97e940c2377 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ceef800c47564c23a140b7f0e2382a02', 'request_id': 'req-1070025b-2755-47b9-8c54-e97e940c2377'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:58.812821 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-1070025b-2755-47b9-8c54-e97e940c2377 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ceef800c47564c23a140b7f0e2382a02', 'request_id': 'req-1070025b-2755-47b9-8c54-e97e940c2377'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:58.833607 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-9678043d-1623-4172-be34-50c7dab5d067 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47795f82d672404485e450f41a6318f8'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:58.840132 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-1070025b-2755-47b9-8c54-e97e940c2377 admin admin] The token cache is being invalidate because project ceef800c47564c23a140b7f0e2382a02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:47:58.840435 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-1070025b-2755-47b9-8c54-e97e940c2377 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:58.841329 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-1070025b-2755-47b9-8c54-e97e940c2377 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:58.842943 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1117/2233] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:47:58 2022] DELETE /identity/v3/projects/ceef800c47564c23a140b7f0e2382a02 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:58.846955 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1070025b-2755-47b9-8c54-e97e940c2377 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:58.848410 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-1070025b-2755-47b9-8c54-e97e940c2377 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:58.904349 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-9678043d-1623-4172-be34-50c7dab5d067 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47795f82d672404485e450f41a6318f8'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:58.913036 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-9678043d-1623-4172-be34-50c7dab5d067 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47795f82d672404485e450f41a6318f8'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:58.916343 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1070025b-2755-47b9-8c54-e97e940c2377 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:58.922015 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d1431c-3e08-4f12-affc-fc643fb5222a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:58.922902 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1431c-3e08-4f12-affc-fc643fb5222a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:58.923187 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1431c-3e08-4f12-affc-fc643fb5222a admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:58.923466 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1431c-3e08-4f12-affc-fc643fb5222a admin admin] PATH_INFO: `/v3/projects/6492bf2a54994ad5a28c7de3eebfbbce` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:58.925274 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d1431c-3e08-4f12-affc-fc643fb5222a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6492bf2a54994ad5a28c7de3eebfbbce)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:58.926313 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-9678043d-1623-4172-be34-50c7dab5d067 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47795f82d672404485e450f41a6318f8'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:58.927254 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-9678043d-1623-4172-be34-50c7dab5d067 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47795f82d672404485e450f41a6318f8'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:58.927745 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d1431c-3e08-4f12-affc-fc643fb5222a admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:58.934306 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1117/2234] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:47:58 2022] DELETE /identity/v3/users/47795f82d672404485e450f41a6318f8 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:47:58.936725 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac61c5e9-eff7-48b4-b4e8-9478e8708d57 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:58.936725 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac61c5e9-eff7-48b4-b4e8-9478e8708d57 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:58.936725 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac61c5e9-eff7-48b4-b4e8-9478e8708d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:58.965549 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-ac61c5e9-eff7-48b4-b4e8-9478e8708d57 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:58.974187 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-ac61c5e9-eff7-48b4-b4e8-9478e8708d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:59.033076 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:59.051353 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1118/2235] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:59.064936 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51222d92-11a2-4511-a03f-bf02cdeca793 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:59.065437 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51222d92-11a2-4511-a03f-bf02cdeca793 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:59.065806 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51222d92-11a2-4511-a03f-bf02cdeca793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:59.085582 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-49d1431c-3e08-4f12-affc-fc643fb5222a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6492bf2a54994ad5a28c7de3eebfbbce', 'request_id': 'req-49d1431c-3e08-4f12-affc-fc643fb5222a'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:59.085866 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-51222d92-11a2-4511-a03f-bf02cdeca793 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:59.086864 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-51222d92-11a2-4511-a03f-bf02cdeca793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:59.094849 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-49d1431c-3e08-4f12-affc-fc643fb5222a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6492bf2a54994ad5a28c7de3eebfbbce', 'request_id': 'req-49d1431c-3e08-4f12-affc-fc643fb5222a'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:59.164288 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1119/2236] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:47:59.169279 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51222d92-11a2-4511-a03f-bf02cdeca793 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:59.178681 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51222d92-11a2-4511-a03f-bf02cdeca793 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:59.185515 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c1b35-2ab9-485a-aa50-6922bd895d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:59.185515 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c1b35-2ab9-485a-aa50-6922bd895d45 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:59.185515 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c1b35-2ab9-485a-aa50-6922bd895d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:59.185515 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c1b35-2ab9-485a-aa50-6922bd895d45 admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:59.186143 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56c1b35-2ab9-485a-aa50-6922bd895d45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:47:59.186143 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56c1b35-2ab9-485a-aa50-6922bd895d45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:59.187470 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56c1b35-2ab9-485a-aa50-6922bd895d45 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:59.204135 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1120/2237] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 16:47:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:47:59.204322 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-49d1431c-3e08-4f12-affc-fc643fb5222a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6492bf2a54994ad5a28c7de3eebfbbce', 'request_id': 'req-49d1431c-3e08-4f12-affc-fc643fb5222a'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:59.213874 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c1b35-2ab9-485a-aa50-6922bd895d45 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:59.221330 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-49d1431c-3e08-4f12-affc-fc643fb5222a admin admin] The token cache is being invalidate because project 6492bf2a54994ad5a28c7de3eebfbbce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:47:59.223899 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-49d1431c-3e08-4f12-affc-fc643fb5222a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:59.223899 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-49d1431c-3e08-4f12-affc-fc643fb5222a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:59.227543 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1118/2238] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:47:58 2022] DELETE /identity/v3/projects/6492bf2a54994ad5a28c7de3eebfbbce => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:47:59.227892 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c1b35-2ab9-485a-aa50-6922bd895d45 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:59.233500 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9ca45f-c919-4beb-a58f-44911dcacf40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:59.234095 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9ca45f-c919-4beb-a58f-44911dcacf40 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:59.234384 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9ca45f-c919-4beb-a58f-44911dcacf40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:59.234662 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9ca45f-c919-4beb-a58f-44911dcacf40 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:59.235563 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9ca45f-c919-4beb-a58f-44911dcacf40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:59.237742 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9ca45f-c919-4beb-a58f-44911dcacf40 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:59.239531 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdf9589-a478-44c5-86a8-b5f6c0be012c None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:59.239815 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdf9589-a478-44c5-86a8-b5f6c0be012c None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:59.240101 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdf9589-a478-44c5-86a8-b5f6c0be012c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:59.281308 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-8fdf9589-a478-44c5-86a8-b5f6c0be012c None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:59.288569 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-8fdf9589-a478-44c5-86a8-b5f6c0be012c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:59.307585 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1119/2239] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:59.310623 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:59.313603 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda59633-6799-49f6-8c67-986cc389ae5e None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:59.317219 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda59633-6799-49f6-8c67-986cc389ae5e None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:59.317219 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda59633-6799-49f6-8c67-986cc389ae5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:59.345065 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-fda59633-6799-49f6-8c67-986cc389ae5e None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:59.353740 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-fda59633-6799-49f6-8c67-986cc389ae5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:59.364885 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1120/2240] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:47:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:59.369362 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda59633-6799-49f6-8c67-986cc389ae5e None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:59.380075 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1121/2241] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:47:59 2022] POST /identity/v3/projects => generated 373 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:47:59.383831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9ca45f-c919-4beb-a58f-44911dcacf40 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:59.386387 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-0f9ca45f-c919-4beb-a58f-44911dcacf40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:59.409488 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda59633-6799-49f6-8c67-986cc389ae5e None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:59.504886 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9ca45f-c919-4beb-a58f-44911dcacf40 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:59.516636 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd90a41-a45f-4a14-9cc8-12bf991ac223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:59.517341 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd90a41-a45f-4a14-9cc8-12bf991ac223 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:59.517623 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd90a41-a45f-4a14-9cc8-12bf991ac223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:59.517912 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd90a41-a45f-4a14-9cc8-12bf991ac223 admin admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:59.519605 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd90a41-a45f-4a14-9cc8-12bf991ac223 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:47:59.519915 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd90a41-a45f-4a14-9cc8-12bf991ac223 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:59.521568 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd90a41-a45f-4a14-9cc8-12bf991ac223 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:59.523876 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89994ffe-889b-419d-89b4-09488f90c5b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:59.524580 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89994ffe-889b-419d-89b4-09488f90c5b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:59.524860 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89994ffe-889b-419d-89b4-09488f90c5b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:59.525132 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89994ffe-889b-419d-89b4-09488f90c5b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:59.526315 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89994ffe-889b-419d-89b4-09488f90c5b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:59.528968 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89994ffe-889b-419d-89b4-09488f90c5b5 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:59.551873 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1121/2242] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 16:47:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:47:59.555773 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd90a41-a45f-4a14-9cc8-12bf991ac223 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:59.570199 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd90a41-a45f-4a14-9cc8-12bf991ac223 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:59.580758 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32398da-57de-41a0-8d66-e1681f6b8e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:59.582212 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32398da-57de-41a0-8d66-e1681f6b8e19 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:59.582617 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32398da-57de-41a0-8d66-e1681f6b8e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:59.583004 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32398da-57de-41a0-8d66-e1681f6b8e19 admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:59.584163 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32398da-57de-41a0-8d66-e1681f6b8e19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:59.586240 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32398da-57de-41a0-8d66-e1681f6b8e19 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:59.677275 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1122/2243] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:47:59 2022] POST /identity/v3/users => generated 287 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:47:59.677275 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89994ffe-889b-419d-89b4-09488f90c5b5 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:59.677275 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-89994ffe-889b-419d-89b4-09488f90c5b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:59.689844 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89994ffe-889b-419d-89b4-09488f90c5b5 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:59.694483 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f61122-a11c-49c8-960a-34ef5227adc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:59.695798 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f61122-a11c-49c8-960a-34ef5227adc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:59.696544 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f61122-a11c-49c8-960a-34ef5227adc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:59.697513 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f61122-a11c-49c8-960a-34ef5227adc0 admin admin] PATH_INFO: `/v3/projects/d73e1d57d4a04a1396448a0d22c96066` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:59.698984 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f61122-a11c-49c8-960a-34ef5227adc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d73e1d57d4a04a1396448a0d22c96066)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:59.701278 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f61122-a11c-49c8-960a-34ef5227adc0 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:59.705010 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1122/2244] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:47:59 2022] POST /identity/v3/projects => generated 409 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:47:59.705010 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32398da-57de-41a0-8d66-e1681f6b8e19 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:59.767740 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32398da-57de-41a0-8d66-e1681f6b8e19 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:59.772635 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388a81e1-e1b9-411e-833f-5a71aa2ab9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:59.773317 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a81e1-e1b9-411e-833f-5a71aa2ab9ab admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:59.773613 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a81e1-e1b9-411e-833f-5a71aa2ab9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:59.773888 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a81e1-e1b9-411e-833f-5a71aa2ab9ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:59.774649 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388a81e1-e1b9-411e-833f-5a71aa2ab9ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:59.777106 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388a81e1-e1b9-411e-833f-5a71aa2ab9ab admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:59.787400 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1123/2245] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:59 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:47:59.788240 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388a81e1-e1b9-411e-833f-5a71aa2ab9ab admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:59.799412 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388a81e1-e1b9-411e-833f-5a71aa2ab9ab admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:59.804372 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fe03b1-ba32-4bb7-bef6-d3c713b45cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:59.804986 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe03b1-ba32-4bb7-bef6-d3c713b45cb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:59.805268 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe03b1-ba32-4bb7-bef6-d3c713b45cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:59.805544 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe03b1-ba32-4bb7-bef6-d3c713b45cb4 admin admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:59.806276 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fe03b1-ba32-4bb7-bef6-d3c713b45cb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:59.808285 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fe03b1-ba32-4bb7-bef6-d3c713b45cb4 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:59.871088 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b2f61122-a11c-49c8-960a-34ef5227adc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd73e1d57d4a04a1396448a0d22c96066', 'request_id': 'req-b2f61122-a11c-49c8-960a-34ef5227adc0'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:59.876878 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b2f61122-a11c-49c8-960a-34ef5227adc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd73e1d57d4a04a1396448a0d22c96066', 'request_id': 'req-b2f61122-a11c-49c8-960a-34ef5227adc0'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:59.891038 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1124/2246] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:47:59 2022] POST /identity/v3/users => generated 305 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:47:59.892478 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fe03b1-ba32-4bb7-bef6-d3c713b45cb4 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:59.895793 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b2f61122-a11c-49c8-960a-34ef5227adc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd73e1d57d4a04a1396448a0d22c96066', 'request_id': 'req-b2f61122-a11c-49c8-960a-34ef5227adc0'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:47:59.905736 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fe03b1-ba32-4bb7-bef6-d3c713b45cb4 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:59.968875 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78095c77-aca8-4744-a6e4-9d0368e5cdcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:59.969608 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78095c77-aca8-4744-a6e4-9d0368e5cdcf admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:59.969912 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78095c77-aca8-4744-a6e4-9d0368e5cdcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:59.970197 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78095c77-aca8-4744-a6e4-9d0368e5cdcf admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:59.971001 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78095c77-aca8-4744-a6e4-9d0368e5cdcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:59.972723 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78095c77-aca8-4744-a6e4-9d0368e5cdcf admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:59.983596 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1125/2247] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:59 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:47:59.984443 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78095c77-aca8-4744-a6e4-9d0368e5cdcf admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:59.996140 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78095c77-aca8-4744-a6e4-9d0368e5cdcf admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:00.002150 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe1dfc8-685d-4087-bc32-9f67a5c50d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:00.007778 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1dfc8-685d-4087-bc32-9f67a5c50d09 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:00.007778 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1dfc8-685d-4087-bc32-9f67a5c50d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:00.007778 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1dfc8-685d-4087-bc32-9f67a5c50d09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:00.007778 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe1dfc8-685d-4087-bc32-9f67a5c50d09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:00.007778 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe1dfc8-685d-4087-bc32-9f67a5c50d09 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:00.017572 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1126/2248] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:47:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:48:00.018499 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe1dfc8-685d-4087-bc32-9f67a5c50d09 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:00.029301 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe1dfc8-685d-4087-bc32-9f67a5c50d09 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:00.035155 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c93bdc-90f0-47a9-87ac-89a173736b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:00.040548 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b2f61122-a11c-49c8-960a-34ef5227adc0 admin admin] The token cache is being invalidate because project d73e1d57d4a04a1396448a0d22c96066 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:48:00.041356 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c93bdc-90f0-47a9-87ac-89a173736b0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:00.041498 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b2f61122-a11c-49c8-960a-34ef5227adc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:00.041745 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c93bdc-90f0-47a9-87ac-89a173736b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:00.041745 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c93bdc-90f0-47a9-87ac-89a173736b0f admin admin] PATH_INFO: `/v3/projects/df11ce888d594e6c8ae450ac62ef5be9/users/e3a67de9df634a65a9572636c7094b95/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:00.041950 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b2f61122-a11c-49c8-960a-34ef5227adc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:00.041950 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1123/2249] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:47:59 2022] DELETE /identity/v3/projects/d73e1d57d4a04a1396448a0d22c96066 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:00.041950 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f61122-a11c-49c8-960a-34ef5227adc0 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:00.041950 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-b2f61122-a11c-49c8-960a-34ef5227adc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:00.065088 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f61122-a11c-49c8-960a-34ef5227adc0 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:00.065088 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfbd9d4-5ec5-4360-b3c1-58173cc2f9f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:00.065088 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfbd9d4-5ec5-4360-b3c1-58173cc2f9f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:00.065088 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfbd9d4-5ec5-4360-b3c1-58173cc2f9f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:00.065590 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfbd9d4-5ec5-4360-b3c1-58173cc2f9f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:00.065590 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfbd9d4-5ec5-4360-b3c1-58173cc2f9f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:00.065798 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfbd9d4-5ec5-4360-b3c1-58173cc2f9f0 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:00.074745 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c93bdc-90f0-47a9-87ac-89a173736b0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=df11ce888d594e6c8ae450ac62ef5be9, user_id=e3a67de9df634a65a9572636c7094b95, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:00.077931 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c93bdc-90f0-47a9-87ac-89a173736b0f admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:00.081819 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1124/2250] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:48:00 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:48:00.081819 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a10dbd-5719-4a93-b8a9-67fc42400b1e None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:00.081819 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a10dbd-5719-4a93-b8a9-67fc42400b1e None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:00.081819 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a10dbd-5719-4a93-b8a9-67fc42400b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:00.089923 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:00.101993 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-98a10dbd-5719-4a93-b8a9-67fc42400b1e None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:00.103209 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-98a10dbd-5719-4a93-b8a9-67fc42400b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:00.111565 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1125/2251] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:48:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:00.112319 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a10dbd-5719-4a93-b8a9-67fc42400b1e None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:00.121780 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a10dbd-5719-4a93-b8a9-67fc42400b1e None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:00.126469 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e05dae-af6f-41c1-8281-e29f3e250f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:00.127353 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e05dae-af6f-41c1-8281-e29f3e250f41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:00.127666 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e05dae-af6f-41c1-8281-e29f3e250f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:00.127969 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e05dae-af6f-41c1-8281-e29f3e250f41 admin admin] PATH_INFO: `/v3/projects/0b38e18678f445fd9b440e9f36af4888/users/a1b2cdacd05f476ca8048238a69fa73d/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:00.143118 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e05dae-af6f-41c1-8281-e29f3e250f41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b38e18678f445fd9b440e9f36af4888, user_id=a1b2cdacd05f476ca8048238a69fa73d, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:00.145832 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e05dae-af6f-41c1-8281-e29f3e250f41 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:00.176354 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1127/2252] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:48:00 2022] PUT /identity/v3/projects/df11ce888d594e6c8ae450ac62ef5be9/users/e3a67de9df634a65a9572636c7094b95/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:00.177746 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a072bf3-3a9c-46ff-9def-4967c803b62e None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:00.178040 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a072bf3-3a9c-46ff-9def-4967c803b62e None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:00.178374 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a072bf3-3a9c-46ff-9def-4967c803b62e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:00.197161 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-0a072bf3-3a9c-46ff-9def-4967c803b62e None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:00.199354 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-0a072bf3-3a9c-46ff-9def-4967c803b62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:00.252292 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1126/2253] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:48:00 2022] PUT /identity/v3/projects/0b38e18678f445fd9b440e9f36af4888/users/a1b2cdacd05f476ca8048238a69fa73d/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:00.254268 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4edc8d-083b-4dbb-86c5-a0cc787ec67b None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:00.254728 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4edc8d-083b-4dbb-86c5-a0cc787ec67b None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:00.255165 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4edc8d-083b-4dbb-86c5-a0cc787ec67b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:00.269072 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1128/2254] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:48:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:00.270497 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688659e-e262-4888-9b9b-871dcdc6e738 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:00.270795 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688659e-e262-4888-9b9b-871dcdc6e738 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:00.271299 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688659e-e262-4888-9b9b-871dcdc6e738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:00.291651 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-6c4edc8d-083b-4dbb-86c5-a0cc787ec67b None None] MFA Rules not processed for user `e3a67de9df634a65a9572636c7094b95`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:00.293533 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-6c4edc8d-083b-4dbb-86c5-a0cc787ec67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:00.314053 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-a688659e-e262-4888-9b9b-871dcdc6e738 None None] MFA Rules not processed for user `a1b2cdacd05f476ca8048238a69fa73d`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:00.315363 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-a688659e-e262-4888-9b9b-871dcdc6e738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:00.411648 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1127/2255] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:48:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:00.415815 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1129/2256] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:48:00 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:00.424809 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4edc8d-083b-4dbb-86c5-a0cc787ec67b None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:00.425021 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a688659e-e262-4888-9b9b-871dcdc6e738 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:00.425021 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-a688659e-e262-4888-9b9b-871dcdc6e738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:00.515709 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a688659e-e262-4888-9b9b-871dcdc6e738 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:00.519285 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a16a575-0f20-4be5-a95e-e269a550a1b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:00.519980 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a16a575-0f20-4be5-a95e-e269a550a1b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:00.520269 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a16a575-0f20-4be5-a95e-e269a550a1b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:00.520550 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a16a575-0f20-4be5-a95e-e269a550a1b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:00.521726 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4edc8d-083b-4dbb-86c5-a0cc787ec67b None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:00.526763 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac6c642-604b-4f3c-a182-749ba0422277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:00.527894 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac6c642-604b-4f3c-a182-749ba0422277 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:00.528330 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac6c642-604b-4f3c-a182-749ba0422277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:00.528819 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac6c642-604b-4f3c-a182-749ba0422277 admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:00.529661 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac6c642-604b-4f3c-a182-749ba0422277 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:48:00.529993 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac6c642-604b-4f3c-a182-749ba0422277 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:00.530113 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a16a575-0f20-4be5-a95e-e269a550a1b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:00.531462 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a16a575-0f20-4be5-a95e-e269a550a1b5 service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:00.532032 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac6c642-604b-4f3c-a182-749ba0422277 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:00.541367 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1130/2257] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:48:00 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:48:00.541930 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a16a575-0f20-4be5-a95e-e269a550a1b5 service cinder] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:00.553351 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a16a575-0f20-4be5-a95e-e269a550a1b5 service cinder] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:00.553911 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1128/2258] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 16:48:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:48:00.555024 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac6c642-604b-4f3c-a182-749ba0422277 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:00.576931 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab9a60-4122-46c2-8046-380354e50a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:00.577664 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:DELETE=1,keystone:SELECT=95,keystone:INSERT=4 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:00.580300 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab9a60-4122-46c2-8046-380354e50a6d admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:00.580656 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab9a60-4122-46c2-8046-380354e50a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:00.580951 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab9a60-4122-46c2-8046-380354e50a6d admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:00.581684 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ab9a60-4122-46c2-8046-380354e50a6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:00.582796 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac6c642-604b-4f3c-a182-749ba0422277 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:00.583893 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ab9a60-4122-46c2-8046-380354e50a6d admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:00.594773 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f5e4b9-2112-489e-ba88-025c24254985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:00.595794 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5e4b9-2112-489e-ba88-025c24254985 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:00.596083 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5e4b9-2112-489e-ba88-025c24254985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:00.596430 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5e4b9-2112-489e-ba88-025c24254985 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:00.597156 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f5e4b9-2112-489e-ba88-025c24254985 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:00.600462 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f5e4b9-2112-489e-ba88-025c24254985 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:00.739704 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1131/2259] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:48:00 2022] POST /identity/v3/projects => generated 409 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:48:00.743957 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab9a60-4122-46c2-8046-380354e50a6d admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:00.753142 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1129/2260] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:48:00 2022] POST /identity/v3/projects => generated 379 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:48:00.754636 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f5e4b9-2112-489e-ba88-025c24254985 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:00.844929 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab9a60-4122-46c2-8046-380354e50a6d admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:00.867918 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f5e4b9-2112-489e-ba88-025c24254985 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:00.873252 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19a781-16c2-4d19-ae77-eec6e796d2d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:00.873945 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19a781-16c2-4d19-ae77-eec6e796d2d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:00.874328 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19a781-16c2-4d19-ae77-eec6e796d2d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:00.874498 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19a781-16c2-4d19-ae77-eec6e796d2d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:00.875542 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a19a781-16c2-4d19-ae77-eec6e796d2d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:00.877488 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a19a781-16c2-4d19-ae77-eec6e796d2d6 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:00.907565 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8702a3-bf07-40fd-833d-0277ebd0be06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:00.908355 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8702a3-bf07-40fd-833d-0277ebd0be06 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:00.913886 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8702a3-bf07-40fd-833d-0277ebd0be06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:00.913886 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8702a3-bf07-40fd-833d-0277ebd0be06 admin admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:00.913886 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8702a3-bf07-40fd-833d-0277ebd0be06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:00.913886 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8702a3-bf07-40fd-833d-0277ebd0be06 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:01.003472 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1130/2261] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:48:00 2022] POST /identity/v3/users => generated 290 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:48:01.007744 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19a781-16c2-4d19-ae77-eec6e796d2d6 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:01.021003 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19a781-16c2-4d19-ae77-eec6e796d2d6 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:01.026125 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1a03a9-4711-4e3e-ae20-1cfa23ccce8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:01.026903 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a03a9-4711-4e3e-ae20-1cfa23ccce8f admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:01.027185 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a03a9-4711-4e3e-ae20-1cfa23ccce8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:01.027573 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a03a9-4711-4e3e-ae20-1cfa23ccce8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:01.028498 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1a03a9-4711-4e3e-ae20-1cfa23ccce8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:01.030190 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1a03a9-4711-4e3e-ae20-1cfa23ccce8f admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:01.038547 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1132/2262] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:48:00 2022] POST /identity/v3/users => generated 311 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:48:01.043848 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8702a3-bf07-40fd-833d-0277ebd0be06 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:01.045080 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1131/2263] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:48:01 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:48:01.049081 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1a03a9-4711-4e3e-ae20-1cfa23ccce8f admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:01.055431 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8702a3-bf07-40fd-833d-0277ebd0be06 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:01.060824 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1a03a9-4711-4e3e-ae20-1cfa23ccce8f admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:01.064793 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5149ea65-3a9a-41e3-9977-c858a64b5dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:01.065715 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5149ea65-3a9a-41e3-9977-c858a64b5dd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:01.066013 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5149ea65-3a9a-41e3-9977-c858a64b5dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:01.066153 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe101878-8039-4e11-88c8-a4636dd4f7ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:01.066951 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5149ea65-3a9a-41e3-9977-c858a64b5dd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:01.067082 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe101878-8039-4e11-88c8-a4636dd4f7ad admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:01.067307 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe101878-8039-4e11-88c8-a4636dd4f7ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:01.067665 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe101878-8039-4e11-88c8-a4636dd4f7ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:01.067783 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5149ea65-3a9a-41e3-9977-c858a64b5dd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:01.068404 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe101878-8039-4e11-88c8-a4636dd4f7ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:01.069992 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5149ea65-3a9a-41e3-9977-c858a64b5dd5 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:01.070120 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe101878-8039-4e11-88c8-a4636dd4f7ad admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:01.091031 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1132/2264] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:48:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:48:01.091199 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1133/2265] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:48:01 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:48:01.091294 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe101878-8039-4e11-88c8-a4636dd4f7ad admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:01.092007 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5149ea65-3a9a-41e3-9977-c858a64b5dd5 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:01.108524 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5149ea65-3a9a-41e3-9977-c858a64b5dd5 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:01.110635 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe101878-8039-4e11-88c8-a4636dd4f7ad admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:01.111466 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff537b8c-8510-4018-826a-99d91de46745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:01.112515 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff537b8c-8510-4018-826a-99d91de46745 admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:01.112896 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff537b8c-8510-4018-826a-99d91de46745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:01.113268 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff537b8c-8510-4018-826a-99d91de46745 admin admin] PATH_INFO: `/v3/projects/1c7ab308903c4e7c843efe59d27d128f/users/ce62854017b04fc7a1217e33f45962ee/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:01.118608 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce038c5-b7a2-42da-be18-ee5042a8c6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:01.121834 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce038c5-b7a2-42da-be18-ee5042a8c6e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:01.121834 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce038c5-b7a2-42da-be18-ee5042a8c6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:01.121834 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce038c5-b7a2-42da-be18-ee5042a8c6e6 admin admin] PATH_INFO: `/v3/projects/eb51ba65f83a45148de31357b8f30a4d/users/30d0a530d7504f62a5fd791efe5bf3a3/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:01.140628 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff537b8c-8510-4018-826a-99d91de46745 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c7ab308903c4e7c843efe59d27d128f, user_id=ce62854017b04fc7a1217e33f45962ee, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:01.143665 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff537b8c-8510-4018-826a-99d91de46745 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:01.149467 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce038c5-b7a2-42da-be18-ee5042a8c6e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb51ba65f83a45148de31357b8f30a4d, user_id=30d0a530d7504f62a5fd791efe5bf3a3, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:01.155974 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce038c5-b7a2-42da-be18-ee5042a8c6e6 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:01.288635 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1134/2266] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:48:01 2022] PUT /identity/v3/projects/1c7ab308903c4e7c843efe59d27d128f/users/ce62854017b04fc7a1217e33f45962ee/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:48:01.292926 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:01.293544 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1133/2267] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:48:01 2022] PUT /identity/v3/projects/eb51ba65f83a45148de31357b8f30a4d/users/30d0a530d7504f62a5fd791efe5bf3a3/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:48:01.303161 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff537b8c-8510-4018-826a-99d91de46745 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:01.305202 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23c0c6-e29a-413a-b7c6-35fea755bcdb None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:01.305498 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23c0c6-e29a-413a-b7c6-35fea755bcdb None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:01.305841 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23c0c6-e29a-413a-b7c6-35fea755bcdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:01.336622 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-2e23c0c6-e29a-413a-b7c6-35fea755bcdb None None] MFA Rules not processed for user `30d0a530d7504f62a5fd791efe5bf3a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:01.338673 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-2e23c0c6-e29a-413a-b7c6-35fea755bcdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:01.392762 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff537b8c-8510-4018-826a-99d91de46745 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:01.421454 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1134/2268] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:48:01 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:01.426225 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e23c0c6-e29a-413a-b7c6-35fea755bcdb None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:01.453923 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca92e4-ec10-43a9-9691-2faeb7dc6a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:01.454836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca92e4-ec10-43a9-9691-2faeb7dc6a99 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:01.455154 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca92e4-ec10-43a9-9691-2faeb7dc6a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:01.455446 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca92e4-ec10-43a9-9691-2faeb7dc6a99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:01.456215 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ca92e4-ec10-43a9-9691-2faeb7dc6a99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:01.457846 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ca92e4-ec10-43a9-9691-2faeb7dc6a99 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:01.486395 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1135/2269] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:48:01 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:48:01.487400 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca92e4-ec10-43a9-9691-2faeb7dc6a99 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:01.502070 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca92e4-ec10-43a9-9691-2faeb7dc6a99 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:01.506488 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d311b-3317-498c-8013-94da8479cc67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:01.507552 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d311b-3317-498c-8013-94da8479cc67 admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:01.507909 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d311b-3317-498c-8013-94da8479cc67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:01.508604 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d311b-3317-498c-8013-94da8479cc67 admin admin] PATH_INFO: `/v3/projects/1c7ab308903c4e7c843efe59d27d128f/users/ce62854017b04fc7a1217e33f45962ee/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:01.513291 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28d311b-3317-498c-8013-94da8479cc67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c7ab308903c4e7c843efe59d27d128f, user_id=ce62854017b04fc7a1217e33f45962ee, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:01.516461 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28d311b-3317-498c-8013-94da8479cc67 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:01.527799 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e23c0c6-e29a-413a-b7c6-35fea755bcdb None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:01.538462 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1136/2270] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:48:01 2022] PUT /identity/v3/projects/1c7ab308903c4e7c843efe59d27d128f/users/ce62854017b04fc7a1217e33f45962ee/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:48:01.549584 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf03dbd9-4e2a-40e9-a684-7c21eb70c3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:01.550349 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03dbd9-4e2a-40e9-a684-7c21eb70c3f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:01.550644 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03dbd9-4e2a-40e9-a684-7c21eb70c3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:01.550929 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03dbd9-4e2a-40e9-a684-7c21eb70c3f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:01.551061 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8b4118-9c79-46ed-946f-3e8bbe3202c9 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:01.551704 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8b4118-9c79-46ed-946f-3e8bbe3202c9 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:01.551704 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8b4118-9c79-46ed-946f-3e8bbe3202c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:01.551874 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf03dbd9-4e2a-40e9-a684-7c21eb70c3f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:01.553649 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf03dbd9-4e2a-40e9-a684-7c21eb70c3f8 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:01.591725 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-eb8b4118-9c79-46ed-946f-3e8bbe3202c9 None None] MFA Rules not processed for user `ce62854017b04fc7a1217e33f45962ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:01.593008 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-eb8b4118-9c79-46ed-946f-3e8bbe3202c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:01.678781 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1137/2271] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:48:01 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:01.690387 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8b4118-9c79-46ed-946f-3e8bbe3202c9 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:01.732451 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1135/2272] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:48:01 2022] POST /identity/v3/projects => generated 381 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:48:01.734118 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf03dbd9-4e2a-40e9-a684-7c21eb70c3f8 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:01.765424 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8b4118-9c79-46ed-946f-3e8bbe3202c9 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:01.818452 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf03dbd9-4e2a-40e9-a684-7c21eb70c3f8 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:01.828937 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba50cd6-328c-403b-9198-037d4efc8f7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:01.829610 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba50cd6-328c-403b-9198-037d4efc8f7e service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:01.829895 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba50cd6-328c-403b-9198-037d4efc8f7e service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:01.830174 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba50cd6-328c-403b-9198-037d4efc8f7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:01.838542 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b3114d-db85-4d5e-9303-f8ea15333969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:01.839950 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b3114d-db85-4d5e-9303-f8ea15333969 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:01.840391 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b3114d-db85-4d5e-9303-f8ea15333969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:01.841236 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b3114d-db85-4d5e-9303-f8ea15333969 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:01.842314 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b3114d-db85-4d5e-9303-f8ea15333969 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:01.844623 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b3114d-db85-4d5e-9303-f8ea15333969 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:01.897514 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba50cd6-328c-403b-9198-037d4efc8f7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:01.899077 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba50cd6-328c-403b-9198-037d4efc8f7e service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:01.911190 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1138/2273] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:48:01 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:48:01.978110 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1136/2274] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:48:01 2022] POST /identity/v3/users => generated 297 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:48:01.978751 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba50cd6-328c-403b-9198-037d4efc8f7e service cinder] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:01.996742 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba50cd6-328c-403b-9198-037d4efc8f7e service cinder] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:02.008252 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ea130e-a6f4-4d8d-b5eb-66e157106f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:02.008956 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea130e-a6f4-4d8d-b5eb-66e157106f04 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:02.009241 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea130e-a6f4-4d8d-b5eb-66e157106f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:02.009527 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea130e-a6f4-4d8d-b5eb-66e157106f04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:02.012409 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ea130e-a6f4-4d8d-b5eb-66e157106f04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:02.014246 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ea130e-a6f4-4d8d-b5eb-66e157106f04 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:02.037775 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1139/2275] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:48:01 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:48:02.040279 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b3114d-db85-4d5e-9303-f8ea15333969 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:02.063846 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b3114d-db85-4d5e-9303-f8ea15333969 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:02.074786 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1068f10b-83e3-4345-9a9c-6cceb0a8b42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:02.075995 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068f10b-83e3-4345-9a9c-6cceb0a8b42e admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:02.076444 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068f10b-83e3-4345-9a9c-6cceb0a8b42e admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:02.076728 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068f10b-83e3-4345-9a9c-6cceb0a8b42e admin admin] PATH_INFO: `/v3/projects/c371d3651662432cab13225966715e27/users/75178757362f43f69f4883787d0932da/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:02.100647 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1068f10b-83e3-4345-9a9c-6cceb0a8b42e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c371d3651662432cab13225966715e27, user_id=75178757362f43f69f4883787d0932da, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:02.103644 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1068f10b-83e3-4345-9a9c-6cceb0a8b42e admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:02.109928 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:02.213859 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1137/2276] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:48:02 2022] PUT /identity/v3/projects/c371d3651662432cab13225966715e27/users/75178757362f43f69f4883787d0932da/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:02.218228 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ea130e-a6f4-4d8d-b5eb-66e157106f04 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:02.233030 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ea130e-a6f4-4d8d-b5eb-66e157106f04 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:02.305410 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7e4690-b60c-4612-ae3d-ad227aa09f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:02.306090 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e4690-b60c-4612-ae3d-ad227aa09f88 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:02.306374 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e4690-b60c-4612-ae3d-ad227aa09f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:02.306651 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e4690-b60c-4612-ae3d-ad227aa09f88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:02.308086 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7e4690-b60c-4612-ae3d-ad227aa09f88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:02.309739 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7e4690-b60c-4612-ae3d-ad227aa09f88 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:02.322360 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1140/2277] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:48:02 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:48:02.324336 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1068f10b-83e3-4345-9a9c-6cceb0a8b42e admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:02.334625 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1068f10b-83e3-4345-9a9c-6cceb0a8b42e admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:02.339569 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62047ef2-f573-4530-b03b-3559fec6786d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:02.340447 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62047ef2-f573-4530-b03b-3559fec6786d admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:02.340730 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62047ef2-f573-4530-b03b-3559fec6786d admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:02.341007 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62047ef2-f573-4530-b03b-3559fec6786d admin admin] PATH_INFO: `/v3/projects/c371d3651662432cab13225966715e27/users/75178757362f43f69f4883787d0932da/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:02.343193 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62047ef2-f573-4530-b03b-3559fec6786d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c371d3651662432cab13225966715e27, user_id=75178757362f43f69f4883787d0932da, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:02.346550 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62047ef2-f573-4530-b03b-3559fec6786d admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:02.357837 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1138/2278] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:48:02 2022] PUT /identity/v3/projects/c371d3651662432cab13225966715e27/users/75178757362f43f69f4883787d0932da/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:02.374054 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc60eacf-c407-4e7c-ae3c-76319bab122a None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:02.374054 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc60eacf-c407-4e7c-ae3c-76319bab122a None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:02.374054 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc60eacf-c407-4e7c-ae3c-76319bab122a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:02.401479 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-cc60eacf-c407-4e7c-ae3c-76319bab122a None None] MFA Rules not processed for user `75178757362f43f69f4883787d0932da`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:02.402739 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-cc60eacf-c407-4e7c-ae3c-76319bab122a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:02.435040 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62047ef2-f573-4530-b03b-3559fec6786d admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:02.488682 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1141/2279] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:48:02 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:48:02.501818 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc60eacf-c407-4e7c-ae3c-76319bab122a None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:02.503433 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-cc60eacf-c407-4e7c-ae3c-76319bab122a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:02.508352 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62047ef2-f573-4530-b03b-3559fec6786d admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:02.513522 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d365ee7d-a097-4839-8377-86e1c844c87a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:02.514239 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d365ee7d-a097-4839-8377-86e1c844c87a service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:02.514531 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d365ee7d-a097-4839-8377-86e1c844c87a service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:02.514810 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d365ee7d-a097-4839-8377-86e1c844c87a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:02.520886 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc60eacf-c407-4e7c-ae3c-76319bab122a None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:02.524720 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e9aac5-ef13-4738-8fe9-b50e00969211 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:02.525492 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e9aac5-ef13-4738-8fe9-b50e00969211 service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:02.525915 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e9aac5-ef13-4738-8fe9-b50e00969211 service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:02.526188 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e9aac5-ef13-4738-8fe9-b50e00969211 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:02.536176 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e9aac5-ef13-4738-8fe9-b50e00969211 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:02.537722 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e9aac5-ef13-4738-8fe9-b50e00969211 service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:02.550250 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1142/2280] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:48:02 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:48:02.587594 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d365ee7d-a097-4839-8377-86e1c844c87a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:02.589950 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d365ee7d-a097-4839-8377-86e1c844c87a service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:02.600232 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1139/2281] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:48:02 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:48:02.921501 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:03.200275 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:03.828956 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:03.835159 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e9aac5-ef13-4738-8fe9-b50e00969211 service cinder] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:03.921989 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e9aac5-ef13-4738-8fe9-b50e00969211 service cinder] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:03.927520 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9621b2c-19f6-4f3a-94a2-e47ea56397aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:03.928561 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9621b2c-19f6-4f3a-94a2-e47ea56397aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:03.928950 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9621b2c-19f6-4f3a-94a2-e47ea56397aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:03.929302 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9621b2c-19f6-4f3a-94a2-e47ea56397aa admin admin] PATH_INFO: `/v3/users/30d0a530d7504f62a5fd791efe5bf3a3` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:03.931172 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9621b2c-19f6-4f3a-94a2-e47ea56397aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=30d0a530d7504f62a5fd791efe5bf3a3)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:03.934623 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9621b2c-19f6-4f3a-94a2-e47ea56397aa admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:04.028100 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:04.216186 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-e9621b2c-19f6-4f3a-94a2-e47ea56397aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30d0a530d7504f62a5fd791efe5bf3a3'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:04.293944 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-e9621b2c-19f6-4f3a-94a2-e47ea56397aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30d0a530d7504f62a5fd791efe5bf3a3'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:04.294964 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-e9621b2c-19f6-4f3a-94a2-e47ea56397aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30d0a530d7504f62a5fd791efe5bf3a3'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:04.295825 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-e9621b2c-19f6-4f3a-94a2-e47ea56397aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30d0a530d7504f62a5fd791efe5bf3a3'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:04.304464 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-e9621b2c-19f6-4f3a-94a2-e47ea56397aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30d0a530d7504f62a5fd791efe5bf3a3'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:04.326884 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1143/2282] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:48:03 2022] DELETE /identity/v3/users/30d0a530d7504f62a5fd791efe5bf3a3 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:04.331693 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c753c-0211-46ec-8258-d59f7aa2db5e None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:04.332206 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c753c-0211-46ec-8258-d59f7aa2db5e None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:04.332486 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c753c-0211-46ec-8258-d59f7aa2db5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:04.350248 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-211c753c-0211-46ec-8258-d59f7aa2db5e None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:04.351458 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-211c753c-0211-46ec-8258-d59f7aa2db5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:04.426362 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1140/2283] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:48:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:04.440907 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9621b2c-19f6-4f3a-94a2-e47ea56397aa admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:04.447001 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-e9621b2c-19f6-4f3a-94a2-e47ea56397aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:04.512668 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9621b2c-19f6-4f3a-94a2-e47ea56397aa admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:04.517574 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba34d48a-7bdc-438c-843b-69cb480b4d90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:04.518317 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba34d48a-7bdc-438c-843b-69cb480b4d90 service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:04.518668 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba34d48a-7bdc-438c-843b-69cb480b4d90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:04.519017 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba34d48a-7bdc-438c-843b-69cb480b4d90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:04.529503 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba34d48a-7bdc-438c-843b-69cb480b4d90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:04.531187 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba34d48a-7bdc-438c-843b-69cb480b4d90 service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:04.542741 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1144/2284] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:48:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:48:05.247983 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c753c-0211-46ec-8258-d59f7aa2db5e None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:05.247983 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-211c753c-0211-46ec-8258-d59f7aa2db5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:05.272923 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c753c-0211-46ec-8258-d59f7aa2db5e None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:05.278906 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a791731-ad2c-4caa-a180-a1989003f655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:05.279860 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a791731-ad2c-4caa-a180-a1989003f655 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:05.280164 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a791731-ad2c-4caa-a180-a1989003f655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:05.280421 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a791731-ad2c-4caa-a180-a1989003f655 admin admin] PATH_INFO: `/v3/projects/eb51ba65f83a45148de31357b8f30a4d` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:05.281892 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a791731-ad2c-4caa-a180-a1989003f655 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb51ba65f83a45148de31357b8f30a4d)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:05.284148 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a791731-ad2c-4caa-a180-a1989003f655 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:05.856880 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-1a791731-ad2c-4caa-a180-a1989003f655 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb51ba65f83a45148de31357b8f30a4d', 'request_id': 'req-1a791731-ad2c-4caa-a180-a1989003f655'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:05.856880 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-1a791731-ad2c-4caa-a180-a1989003f655 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb51ba65f83a45148de31357b8f30a4d', 'request_id': 'req-1a791731-ad2c-4caa-a180-a1989003f655'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:05.856880 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-1a791731-ad2c-4caa-a180-a1989003f655 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb51ba65f83a45148de31357b8f30a4d', 'request_id': 'req-1a791731-ad2c-4caa-a180-a1989003f655'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:05.859187 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba34d48a-7bdc-438c-843b-69cb480b4d90 service neutron] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:05.859187 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-ba34d48a-7bdc-438c-843b-69cb480b4d90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:05.859187 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba34d48a-7bdc-438c-843b-69cb480b4d90 service neutron] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:05.859723 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fedfc56-1f2d-4aee-b1cb-e6da87838515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:05.859723 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fedfc56-1f2d-4aee-b1cb-e6da87838515 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:05.859723 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fedfc56-1f2d-4aee-b1cb-e6da87838515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:05.859723 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fedfc56-1f2d-4aee-b1cb-e6da87838515 admin admin] PATH_INFO: `/v3/users/e3a67de9df634a65a9572636c7094b95` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:05.860010 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fedfc56-1f2d-4aee-b1cb-e6da87838515 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3a67de9df634a65a9572636c7094b95)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:05.860357 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fedfc56-1f2d-4aee-b1cb-e6da87838515 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:05.931273 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-1a791731-ad2c-4caa-a180-a1989003f655 admin admin] The token cache is being invalidate because project eb51ba65f83a45148de31357b8f30a4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:48:05.933292 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-1a791731-ad2c-4caa-a180-a1989003f655 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:05.933292 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-1a791731-ad2c-4caa-a180-a1989003f655 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:05.934813 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1141/2285] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:48:05 2022] DELETE /identity/v3/projects/eb51ba65f83a45148de31357b8f30a4d => generated 0 bytes in 691 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:05.939513 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a791731-ad2c-4caa-a180-a1989003f655 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:05.941030 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-1a791731-ad2c-4caa-a180-a1989003f655 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:05.955731 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a791731-ad2c-4caa-a180-a1989003f655 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:05.961404 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc87e20b-9407-4e67-a413-d2236c6aa9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:05.962278 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc87e20b-9407-4e67-a413-d2236c6aa9be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:05.962566 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc87e20b-9407-4e67-a413-d2236c6aa9be admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:05.962855 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc87e20b-9407-4e67-a413-d2236c6aa9be admin admin] PATH_INFO: `/v3/users/75178757362f43f69f4883787d0932da` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:05.964803 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc87e20b-9407-4e67-a413-d2236c6aa9be admin admin] RBAC: Authorizing `identity:delete_user(user_id=75178757362f43f69f4883787d0932da)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:05.974910 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc87e20b-9407-4e67-a413-d2236c6aa9be admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:06.061695 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:06.079747 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-2fedfc56-1f2d-4aee-b1cb-e6da87838515 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3a67de9df634a65a9572636c7094b95'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:06.167358 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:06.177242 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-dc87e20b-9407-4e67-a413-d2236c6aa9be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75178757362f43f69f4883787d0932da'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:06.184435 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-2fedfc56-1f2d-4aee-b1cb-e6da87838515 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3a67de9df634a65a9572636c7094b95'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:06.190477 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-2fedfc56-1f2d-4aee-b1cb-e6da87838515 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3a67de9df634a65a9572636c7094b95'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:06.190477 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-2fedfc56-1f2d-4aee-b1cb-e6da87838515 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3a67de9df634a65a9572636c7094b95'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:06.194080 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-2fedfc56-1f2d-4aee-b1cb-e6da87838515 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3a67de9df634a65a9572636c7094b95'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:06.220114 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1145/2286] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:48:05 2022] DELETE /identity/v3/users/e3a67de9df634a65a9572636c7094b95 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:48:06.224188 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f542e43-d5e9-4a0b-8292-3ddf8c17fc6f None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:06.224550 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f542e43-d5e9-4a0b-8292-3ddf8c17fc6f None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:06.224830 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f542e43-d5e9-4a0b-8292-3ddf8c17fc6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:06.255942 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-9f542e43-d5e9-4a0b-8292-3ddf8c17fc6f None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:06.257647 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-9f542e43-d5e9-4a0b-8292-3ddf8c17fc6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:06.284902 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-dc87e20b-9407-4e67-a413-d2236c6aa9be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75178757362f43f69f4883787d0932da'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:06.305514 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-dc87e20b-9407-4e67-a413-d2236c6aa9be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75178757362f43f69f4883787d0932da'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:06.320858 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-dc87e20b-9407-4e67-a413-d2236c6aa9be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75178757362f43f69f4883787d0932da'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:06.321620 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-dc87e20b-9407-4e67-a413-d2236c6aa9be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75178757362f43f69f4883787d0932da'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:06.332466 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1142/2287] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:48:05 2022] DELETE /identity/v3/users/75178757362f43f69f4883787d0932da => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:48:06.352382 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:06.357706 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1146/2288] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:48:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:48:06.368039 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc87e20b-9407-4e67-a413-d2236c6aa9be admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:06.370685 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-dc87e20b-9407-4e67-a413-d2236c6aa9be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:06.456580 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc87e20b-9407-4e67-a413-d2236c6aa9be admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:06.465808 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426f80d1-c4f2-43c2-8055-5e1d96b519e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:06.465808 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f80d1-c4f2-43c2-8055-5e1d96b519e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:06.465808 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f80d1-c4f2-43c2-8055-5e1d96b519e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:06.465808 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f80d1-c4f2-43c2-8055-5e1d96b519e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:06.547225 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426f80d1-c4f2-43c2-8055-5e1d96b519e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:06.548733 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426f80d1-c4f2-43c2-8055-5e1d96b519e7 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:06.560142 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1143/2289] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:48:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:48:06.845586 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f542e43-d5e9-4a0b-8292-3ddf8c17fc6f None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:06.848585 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-9f542e43-d5e9-4a0b-8292-3ddf8c17fc6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:06.860513 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f542e43-d5e9-4a0b-8292-3ddf8c17fc6f None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:06.867782 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe7703-0818-4e68-9d8e-96de251e0cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:06.868560 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe7703-0818-4e68-9d8e-96de251e0cfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:06.868845 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe7703-0818-4e68-9d8e-96de251e0cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:06.870636 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe7703-0818-4e68-9d8e-96de251e0cfd admin admin] PATH_INFO: `/v3/projects/c371d3651662432cab13225966715e27` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:06.872731 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fe7703-0818-4e68-9d8e-96de251e0cfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=c371d3651662432cab13225966715e27)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:06.874840 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fe7703-0818-4e68-9d8e-96de251e0cfd admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:06.978133 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426f80d1-c4f2-43c2-8055-5e1d96b519e7 service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:06.980047 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-426f80d1-c4f2-43c2-8055-5e1d96b519e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:06.996100 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426f80d1-c4f2-43c2-8055-5e1d96b519e7 service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:07.000948 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-a6fe7703-0818-4e68-9d8e-96de251e0cfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c371d3651662432cab13225966715e27', 'request_id': 'req-a6fe7703-0818-4e68-9d8e-96de251e0cfd'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:07.006217 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-a6fe7703-0818-4e68-9d8e-96de251e0cfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c371d3651662432cab13225966715e27', 'request_id': 'req-a6fe7703-0818-4e68-9d8e-96de251e0cfd'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:07.053648 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b788cf4-04a3-496a-8001-5b6db5f52316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:07.054543 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b788cf4-04a3-496a-8001-5b6db5f52316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:07.054831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b788cf4-04a3-496a-8001-5b6db5f52316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:07.055110 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b788cf4-04a3-496a-8001-5b6db5f52316 admin admin] PATH_INFO: `/v3/projects/df11ce888d594e6c8ae450ac62ef5be9` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:07.056335 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b788cf4-04a3-496a-8001-5b6db5f52316 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df11ce888d594e6c8ae450ac62ef5be9)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:07.059202 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b788cf4-04a3-496a-8001-5b6db5f52316 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:07.064210 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-a6fe7703-0818-4e68-9d8e-96de251e0cfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c371d3651662432cab13225966715e27', 'request_id': 'req-a6fe7703-0818-4e68-9d8e-96de251e0cfd'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:07.076910 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-a6fe7703-0818-4e68-9d8e-96de251e0cfd admin admin] The token cache is being invalidate because project c371d3651662432cab13225966715e27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:48:07.077210 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-a6fe7703-0818-4e68-9d8e-96de251e0cfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:07.077875 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-a6fe7703-0818-4e68-9d8e-96de251e0cfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:07.079744 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1147/2290] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:48:06 2022] DELETE /identity/v3/projects/c371d3651662432cab13225966715e27 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:07.084713 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c2959-0b85-4a02-b5da-b438fe12e96f None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:07.085001 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c2959-0b85-4a02-b5da-b438fe12e96f None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:07.085305 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c2959-0b85-4a02-b5da-b438fe12e96f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:07.103702 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-0b8c2959-0b85-4a02-b5da-b438fe12e96f None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:07.105098 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-0b8c2959-0b85-4a02-b5da-b438fe12e96f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:07.113349 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1148/2291] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:48:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:48:07.122947 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a977a8-4ecb-40dc-9a0b-e20a1269fd1a None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:07.126884 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a977a8-4ecb-40dc-9a0b-e20a1269fd1a None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:07.126884 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a977a8-4ecb-40dc-9a0b-e20a1269fd1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:07.141840 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-a8a977a8-4ecb-40dc-9a0b-e20a1269fd1a None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:07.142976 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-a8a977a8-4ecb-40dc-9a0b-e20a1269fd1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:07.158570 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-7b788cf4-04a3-496a-8001-5b6db5f52316 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df11ce888d594e6c8ae450ac62ef5be9', 'request_id': 'req-7b788cf4-04a3-496a-8001-5b6db5f52316'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:07.164072 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-7b788cf4-04a3-496a-8001-5b6db5f52316 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df11ce888d594e6c8ae450ac62ef5be9', 'request_id': 'req-7b788cf4-04a3-496a-8001-5b6db5f52316'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:07.171194 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-7b788cf4-04a3-496a-8001-5b6db5f52316 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df11ce888d594e6c8ae450ac62ef5be9', 'request_id': 'req-7b788cf4-04a3-496a-8001-5b6db5f52316'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:07.202977 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1149/2292] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:48:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:48:07.207652 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a977a8-4ecb-40dc-9a0b-e20a1269fd1a None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:07.217878 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a977a8-4ecb-40dc-9a0b-e20a1269fd1a None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:07.222721 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423ae892-db9a-405a-8bda-4c27825778c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:07.224526 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ae892-db9a-405a-8bda-4c27825778c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:07.224526 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ae892-db9a-405a-8bda-4c27825778c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:07.224526 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ae892-db9a-405a-8bda-4c27825778c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:07.224801 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423ae892-db9a-405a-8bda-4c27825778c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:48:07.225004 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423ae892-db9a-405a-8bda-4c27825778c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:07.226656 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423ae892-db9a-405a-8bda-4c27825778c6 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:07.239193 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1150/2293] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 16:48:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:48:07.243343 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423ae892-db9a-405a-8bda-4c27825778c6 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:07.250359 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-7b788cf4-04a3-496a-8001-5b6db5f52316 admin admin] The token cache is being invalidate because project df11ce888d594e6c8ae450ac62ef5be9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:48:07.250657 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-7b788cf4-04a3-496a-8001-5b6db5f52316 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:07.251320 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-7b788cf4-04a3-496a-8001-5b6db5f52316 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:07.252914 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1144/2294] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:48:06 2022] DELETE /identity/v3/projects/df11ce888d594e6c8ae450ac62ef5be9 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:07.253641 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423ae892-db9a-405a-8bda-4c27825778c6 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:07.257982 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d111ca5-a7a7-4237-b4b8-b95c77595e90 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:07.258351 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d111ca5-a7a7-4237-b4b8-b95c77595e90 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:07.258483 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42baa4be-f07c-485d-ba92-9dc8aaccb145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:07.258794 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d111ca5-a7a7-4237-b4b8-b95c77595e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:07.259177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42baa4be-f07c-485d-ba92-9dc8aaccb145 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:07.259432 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42baa4be-f07c-485d-ba92-9dc8aaccb145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:07.266599 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42baa4be-f07c-485d-ba92-9dc8aaccb145 admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:07.266599 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42baa4be-f07c-485d-ba92-9dc8aaccb145 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:07.266807 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:07.266889 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42baa4be-f07c-485d-ba92-9dc8aaccb145 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:07.281993 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-9d111ca5-a7a7-4237-b4b8-b95c77595e90 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:07.283347 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-9d111ca5-a7a7-4237-b4b8-b95c77595e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:07.299706 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1145/2295] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:48:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:48:07.301228 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc8d38-6bed-43d7-b570-639a4ac55067 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:07.301624 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc8d38-6bed-43d7-b570-639a4ac55067 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:07.301937 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc8d38-6bed-43d7-b570-639a4ac55067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:07.324780 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-d7bc8d38-6bed-43d7-b570-639a4ac55067 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:07.327883 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d7bc8d38-6bed-43d7-b570-639a4ac55067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:07.336004 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1146/2296] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:48:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:07.340410 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bc8d38-6bed-43d7-b570-639a4ac55067 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:07.351387 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bc8d38-6bed-43d7-b570-639a4ac55067 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:07.359772 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1153a89e-8977-4072-a66c-fd23b7bd3845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:07.360508 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1153a89e-8977-4072-a66c-fd23b7bd3845 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:07.361494 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1153a89e-8977-4072-a66c-fd23b7bd3845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:07.361798 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1153a89e-8977-4072-a66c-fd23b7bd3845 admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:07.362896 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1153a89e-8977-4072-a66c-fd23b7bd3845 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:48:07.363209 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1153a89e-8977-4072-a66c-fd23b7bd3845 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:07.365636 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1153a89e-8977-4072-a66c-fd23b7bd3845 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:07.382739 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1147/2297] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 16:48:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:48:07.383887 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1153a89e-8977-4072-a66c-fd23b7bd3845 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:07.398907 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1153a89e-8977-4072-a66c-fd23b7bd3845 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:07.400042 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba60b8-cad8-4921-9dbe-5d079b556d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:07.400732 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba60b8-cad8-4921-9dbe-5d079b556d0e admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:07.401011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba60b8-cad8-4921-9dbe-5d079b556d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:07.401290 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba60b8-cad8-4921-9dbe-5d079b556d0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:07.401989 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaba60b8-cad8-4921-9dbe-5d079b556d0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:07.403952 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaba60b8-cad8-4921-9dbe-5d079b556d0e admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:07.405040 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1151/2298] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:48:07 2022] POST /identity/v3/projects => generated 373 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:48:07.409059 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42baa4be-f07c-485d-ba92-9dc8aaccb145 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:07.410800 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-42baa4be-f07c-485d-ba92-9dc8aaccb145 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:07.491484 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42baa4be-f07c-485d-ba92-9dc8aaccb145 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:07.498281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1ba996-e57e-4893-81f4-89c81c67a417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:07.498959 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ba996-e57e-4893-81f4-89c81c67a417 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:07.499246 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ba996-e57e-4893-81f4-89c81c67a417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:07.499555 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ba996-e57e-4893-81f4-89c81c67a417 admin admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:07.500347 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1ba996-e57e-4893-81f4-89c81c67a417 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:07.502261 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1ba996-e57e-4893-81f4-89c81c67a417 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:07.529133 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1148/2299] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:48:07 2022] POST /identity/v3/projects => generated 373 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:48:07.533367 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba60b8-cad8-4921-9dbe-5d079b556d0e admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:07.544983 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:07.596669 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba60b8-cad8-4921-9dbe-5d079b556d0e admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:07.603109 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea70268-40e4-40cb-a081-9f36b11ac365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:07.604426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea70268-40e4-40cb-a081-9f36b11ac365 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:07.604713 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea70268-40e4-40cb-a081-9f36b11ac365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:07.605143 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea70268-40e4-40cb-a081-9f36b11ac365 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:07.606031 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea70268-40e4-40cb-a081-9f36b11ac365 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:07.608134 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea70268-40e4-40cb-a081-9f36b11ac365 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:07.610854 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1152/2300] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:48:07 2022] POST /identity/v3/users => generated 287 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:48:07.674041 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1ba996-e57e-4893-81f4-89c81c67a417 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:07.685611 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1ba996-e57e-4893-81f4-89c81c67a417 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:07.690380 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0a8d26-df8f-45eb-94fe-eab9390b0dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:07.691335 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0a8d26-df8f-45eb-94fe-eab9390b0dd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:07.691335 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0a8d26-df8f-45eb-94fe-eab9390b0dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:07.691632 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0a8d26-df8f-45eb-94fe-eab9390b0dd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:07.692469 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0a8d26-df8f-45eb-94fe-eab9390b0dd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:07.693986 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0a8d26-df8f-45eb-94fe-eab9390b0dd4 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:07.705776 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1153/2301] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:48:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:48:07.711129 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0a8d26-df8f-45eb-94fe-eab9390b0dd4 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:07.728274 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0a8d26-df8f-45eb-94fe-eab9390b0dd4 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:07.734248 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06203f5-eb4c-4388-8c1e-601dfc510b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:07.736099 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1149/2302] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:48:07 2022] POST /identity/v3/users => generated 287 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:48:07.736220 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06203f5-eb4c-4388-8c1e-601dfc510b5e admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:07.736220 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06203f5-eb4c-4388-8c1e-601dfc510b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:07.736220 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06203f5-eb4c-4388-8c1e-601dfc510b5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:07.736440 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06203f5-eb4c-4388-8c1e-601dfc510b5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:07.738128 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06203f5-eb4c-4388-8c1e-601dfc510b5e admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:07.738653 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea70268-40e4-40cb-a081-9f36b11ac365 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:07.750731 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea70268-40e4-40cb-a081-9f36b11ac365 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:07.751668 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1154/2303] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:48:07 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:48:07.756168 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a1bd45-256c-43b1-9925-96d247f4e2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:07.758846 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a1bd45-256c-43b1-9925-96d247f4e2e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:07.758846 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a1bd45-256c-43b1-9925-96d247f4e2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:07.758846 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a1bd45-256c-43b1-9925-96d247f4e2e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:07.759117 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06203f5-eb4c-4388-8c1e-601dfc510b5e admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:07.759230 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a1bd45-256c-43b1-9925-96d247f4e2e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:07.759988 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a1bd45-256c-43b1-9925-96d247f4e2e0 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:07.768925 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06203f5-eb4c-4388-8c1e-601dfc510b5e admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:07.771270 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1150/2304] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:48:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:48:07.773781 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a5e4f1-435c-4759-b6ab-473e8c49900d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:07.774745 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5e4f1-435c-4759-b6ab-473e8c49900d admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:07.775476 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5e4f1-435c-4759-b6ab-473e8c49900d admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:07.775634 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a1bd45-256c-43b1-9925-96d247f4e2e0 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:07.775724 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5e4f1-435c-4759-b6ab-473e8c49900d admin admin] PATH_INFO: `/v3/projects/8a4186243afe47b5be831c9584eeef6d/users/ea704b4947e8477cb261f3a61c786155/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:07.786611 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a1bd45-256c-43b1-9925-96d247f4e2e0 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:07.792765 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a5e4f1-435c-4759-b6ab-473e8c49900d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a4186243afe47b5be831c9584eeef6d, user_id=ea704b4947e8477cb261f3a61c786155, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:07.793234 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707f89f-76f2-4ba7-b250-873851180445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:07.793914 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707f89f-76f2-4ba7-b250-873851180445 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:07.794295 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707f89f-76f2-4ba7-b250-873851180445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:07.794576 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707f89f-76f2-4ba7-b250-873851180445 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:07.795303 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2707f89f-76f2-4ba7-b250-873851180445 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:07.795592 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a5e4f1-435c-4759-b6ab-473e8c49900d admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:07.796945 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2707f89f-76f2-4ba7-b250-873851180445 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:07.814235 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1151/2305] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:48:07 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:48:07.818643 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707f89f-76f2-4ba7-b250-873851180445 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:07.834190 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707f89f-76f2-4ba7-b250-873851180445 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:07.842454 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6839af42-28a9-46d7-825e-bccebe975faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:07.845020 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6839af42-28a9-46d7-825e-bccebe975faa admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:07.845020 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6839af42-28a9-46d7-825e-bccebe975faa admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:07.845020 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6839af42-28a9-46d7-825e-bccebe975faa admin admin] PATH_INFO: `/v3/projects/b2153f74cf4b4bdcb0aab5b6921ca3ad/users/e548ea934bc14b7e921f7e161c5496cf/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:07.863625 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6839af42-28a9-46d7-825e-bccebe975faa admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2153f74cf4b4bdcb0aab5b6921ca3ad, user_id=e548ea934bc14b7e921f7e161c5496cf, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:07.866592 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6839af42-28a9-46d7-825e-bccebe975faa admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:07.874346 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1155/2306] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:48:07 2022] PUT /identity/v3/projects/8a4186243afe47b5be831c9584eeef6d/users/ea704b4947e8477cb261f3a61c786155/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:48:07.881519 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3faed-b4bc-4e66-ab01-9646cb8196df None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:07.881808 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3faed-b4bc-4e66-ab01-9646cb8196df None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:07.882128 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3faed-b4bc-4e66-ab01-9646cb8196df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:07.900544 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-bfd3faed-b4bc-4e66-ab01-9646cb8196df None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:07.902471 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-bfd3faed-b4bc-4e66-ab01-9646cb8196df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:07.966303 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1152/2307] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:48:07 2022] PUT /identity/v3/projects/b2153f74cf4b4bdcb0aab5b6921ca3ad/users/e548ea934bc14b7e921f7e161c5496cf/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:48:07.972784 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be814f-3c34-45f6-841d-db6a62f327ca None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:07.973096 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be814f-3c34-45f6-841d-db6a62f327ca None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:07.973520 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be814f-3c34-45f6-841d-db6a62f327ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:07.977364 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1156/2308] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:48:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:48:07.990562 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd3faed-b4bc-4e66-ab01-9646cb8196df None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:07.992041 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-bfd3faed-b4bc-4e66-ab01-9646cb8196df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:07.993580 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-66be814f-3c34-45f6-841d-db6a62f327ca None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:07.994994 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-66be814f-3c34-45f6-841d-db6a62f327ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:08.053468 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd3faed-b4bc-4e66-ab01-9646cb8196df None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:08.064489 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1153/2309] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:48:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:48:08.074930 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be814f-3c34-45f6-841d-db6a62f327ca None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:08.076706 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-66be814f-3c34-45f6-841d-db6a62f327ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:08.108232 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcb5683-936c-4c0c-b0f1-6ac6b58c6123 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:08.108939 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb5683-936c-4c0c-b0f1-6ac6b58c6123 service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:08.109285 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb5683-936c-4c0c-b0f1-6ac6b58c6123 service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:08.109490 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb5683-936c-4c0c-b0f1-6ac6b58c6123 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:08.119200 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcb5683-936c-4c0c-b0f1-6ac6b58c6123 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:08.121750 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcb5683-936c-4c0c-b0f1-6ac6b58c6123 service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:08.133012 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1157/2310] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:48:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:48:08.141304 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be814f-3c34-45f6-841d-db6a62f327ca None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:08.145454 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffce8777-565b-4add-a624-e97b732917ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:08.146111 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffce8777-565b-4add-a624-e97b732917ae service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:08.146396 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffce8777-565b-4add-a624-e97b732917ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:08.146671 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffce8777-565b-4add-a624-e97b732917ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:08.157381 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffce8777-565b-4add-a624-e97b732917ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:08.160225 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffce8777-565b-4add-a624-e97b732917ae service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:08.169936 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1154/2311] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:48:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:48:08.605898 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:08.658944 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:09.848060 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b940d1-6939-4ced-9f33-2ba8aabc08d2 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:09.848599 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b940d1-6939-4ced-9f33-2ba8aabc08d2 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:09.848699 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b940d1-6939-4ced-9f33-2ba8aabc08d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:09.881317 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-b8b940d1-6939-4ced-9f33-2ba8aabc08d2 None None] MFA Rules not processed for user `e548ea934bc14b7e921f7e161c5496cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:09.882883 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-b8b940d1-6939-4ced-9f33-2ba8aabc08d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:09.894006 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e9480-3855-4907-bb12-e287b87e5169 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:09.894575 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e9480-3855-4907-bb12-e287b87e5169 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:09.894575 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e9480-3855-4907-bb12-e287b87e5169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:09.950533 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-db6e9480-3855-4907-bb12-e287b87e5169 None None] MFA Rules not processed for user `ea704b4947e8477cb261f3a61c786155`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:09.952637 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-db6e9480-3855-4907-bb12-e287b87e5169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:10.048819 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1158/2312] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:48:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:10.083808 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1155/2313] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:48:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:48:10.084000 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b940d1-6939-4ced-9f33-2ba8aabc08d2 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:10.085069 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6e9480-3855-4907-bb12-e287b87e5169 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:10.085642 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-b8b940d1-6939-4ced-9f33-2ba8aabc08d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:10.098641 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6e9480-3855-4907-bb12-e287b87e5169 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:10.110852 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005f4dd2-5459-4dcd-ac87-7f992a22b4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:10.112100 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005f4dd2-5459-4dcd-ac87-7f992a22b4d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:10.112396 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005f4dd2-5459-4dcd-ac87-7f992a22b4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:10.112677 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005f4dd2-5459-4dcd-ac87-7f992a22b4d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:10.113680 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005f4dd2-5459-4dcd-ac87-7f992a22b4d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:10.118565 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005f4dd2-5459-4dcd-ac87-7f992a22b4d8 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:10.161258 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b940d1-6939-4ced-9f33-2ba8aabc08d2 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:10.165032 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb0e0b9-9e03-4824-bb00-1d0b80b7efc0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:10.165730 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0e0b9-9e03-4824-bb00-1d0b80b7efc0 service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:10.165986 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0e0b9-9e03-4824-bb00-1d0b80b7efc0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:10.166322 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0e0b9-9e03-4824-bb00-1d0b80b7efc0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:10.175397 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb0e0b9-9e03-4824-bb00-1d0b80b7efc0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:10.176884 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb0e0b9-9e03-4824-bb00-1d0b80b7efc0 service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:10.187146 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1159/2314] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:48:10 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:48:10.242102 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:10.252216 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1156/2315] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:48:10 2022] POST /identity/v3/projects => generated 369 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:48:10.254720 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:10.256323 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb0e0b9-9e03-4824-bb00-1d0b80b7efc0 service cinder] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:10.270346 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb0e0b9-9e03-4824-bb00-1d0b80b7efc0 service cinder] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:10.339624 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b253d18d-9c74-4c70-9517-23a22aebb43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:10.340475 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b253d18d-9c74-4c70-9517-23a22aebb43d admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:10.340792 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b253d18d-9c74-4c70-9517-23a22aebb43d admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:10.341074 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b253d18d-9c74-4c70-9517-23a22aebb43d admin admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:10.341780 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b253d18d-9c74-4c70-9517-23a22aebb43d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:10.343772 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b253d18d-9c74-4c70-9517-23a22aebb43d admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:10.526959 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1160/2316] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:48:10 2022] POST /identity/v3/users => generated 291 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:48:10.528491 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005f4dd2-5459-4dcd-ac87-7f992a22b4d8 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:10.543381 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005f4dd2-5459-4dcd-ac87-7f992a22b4d8 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:10.549478 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ee2e3-4a97-42a8-83e1-2e43b29c297a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:10.553134 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ee2e3-4a97-42a8-83e1-2e43b29c297a admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:10.553134 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ee2e3-4a97-42a8-83e1-2e43b29c297a admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:10.553134 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ee2e3-4a97-42a8-83e1-2e43b29c297a admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:10.553134 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4ee2e3-4a97-42a8-83e1-2e43b29c297a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:10.554027 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4ee2e3-4a97-42a8-83e1-2e43b29c297a admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:10.565390 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1157/2317] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:48:10 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:48:10.569874 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b253d18d-9c74-4c70-9517-23a22aebb43d admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:10.581657 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b253d18d-9c74-4c70-9517-23a22aebb43d admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:10.587961 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7ee065-1f20-4e21-ab61-a4cab16e2d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:10.589579 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7ee065-1f20-4e21-ab61-a4cab16e2d77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:10.589871 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7ee065-1f20-4e21-ab61-a4cab16e2d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:10.590154 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7ee065-1f20-4e21-ab61-a4cab16e2d77 admin admin] PATH_INFO: `/v3/projects/f266ab3819b346918360e68e11ae1d8f/users/3e4ad7fc739c4dd6bec17d16969903e4/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:10.612188 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7ee065-1f20-4e21-ab61-a4cab16e2d77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f266ab3819b346918360e68e11ae1d8f, user_id=3e4ad7fc739c4dd6bec17d16969903e4, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:10.615318 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7ee065-1f20-4e21-ab61-a4cab16e2d77 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:10.816050 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1161/2318] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:48:10 2022] PUT /identity/v3/projects/f266ab3819b346918360e68e11ae1d8f/users/3e4ad7fc739c4dd6bec17d16969903e4/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:10.816276 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ee2e3-4a97-42a8-83e1-2e43b29c297a admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:10.822782 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ee2e3-4a97-42a8-83e1-2e43b29c297a admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:10.889704 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778f07f6-0b2b-4f12-b033-b7e987a32efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:10.890389 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778f07f6-0b2b-4f12-b033-b7e987a32efc admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:10.890671 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778f07f6-0b2b-4f12-b033-b7e987a32efc admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:10.890949 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778f07f6-0b2b-4f12-b033-b7e987a32efc admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:10.891714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778f07f6-0b2b-4f12-b033-b7e987a32efc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:10.893338 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778f07f6-0b2b-4f12-b033-b7e987a32efc admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:10.906823 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1158/2319] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:48:10 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:48:10.912774 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7ee065-1f20-4e21-ab61-a4cab16e2d77 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:10.924757 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7ee065-1f20-4e21-ab61-a4cab16e2d77 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:10.929473 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7a6ffd-b3a1-43e5-a3cc-9d9ce438cf0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:10.930398 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7a6ffd-b3a1-43e5-a3cc-9d9ce438cf0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:10.930683 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7a6ffd-b3a1-43e5-a3cc-9d9ce438cf0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:10.930958 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7a6ffd-b3a1-43e5-a3cc-9d9ce438cf0b admin admin] PATH_INFO: `/v3/projects/f266ab3819b346918360e68e11ae1d8f/users/3e4ad7fc739c4dd6bec17d16969903e4/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:10.933147 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7a6ffd-b3a1-43e5-a3cc-9d9ce438cf0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f266ab3819b346918360e68e11ae1d8f, user_id=3e4ad7fc739c4dd6bec17d16969903e4, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:10.935856 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7a6ffd-b3a1-43e5-a3cc-9d9ce438cf0b admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:10.950266 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1162/2320] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:48:10 2022] PUT /identity/v3/projects/f266ab3819b346918360e68e11ae1d8f/users/3e4ad7fc739c4dd6bec17d16969903e4/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:11.875942 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:12.677371 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4a1b2-fe3f-41fc-a35b-ba4202eab0e1 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:12.677871 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4a1b2-fe3f-41fc-a35b-ba4202eab0e1 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:12.677965 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4a1b2-fe3f-41fc-a35b-ba4202eab0e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:12.712674 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-b4c4a1b2-fe3f-41fc-a35b-ba4202eab0e1 None None] MFA Rules not processed for user `3e4ad7fc739c4dd6bec17d16969903e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:12.714304 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-b4c4a1b2-fe3f-41fc-a35b-ba4202eab0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:12.790720 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1159/2321] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:48:12 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:12.803327 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7a6ffd-b3a1-43e5-a3cc-9d9ce438cf0b admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:12.805298 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-4f7a6ffd-b3a1-43e5-a3cc-9d9ce438cf0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:12.866130 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7a6ffd-b3a1-43e5-a3cc-9d9ce438cf0b admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:12.870459 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd14763-73ca-441f-b64d-0123d01a2da0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:12.905865 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd14763-73ca-441f-b64d-0123d01a2da0 service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:12.905865 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd14763-73ca-441f-b64d-0123d01a2da0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:12.905865 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd14763-73ca-441f-b64d-0123d01a2da0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:12.929759 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd14763-73ca-441f-b64d-0123d01a2da0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:12.931207 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd14763-73ca-441f-b64d-0123d01a2da0 service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:12.951770 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1163/2322] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:48:12 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:48:14.456514 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a08454-fdfd-477c-8f91-d25d23cb9462 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:14.457127 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a08454-fdfd-477c-8f91-d25d23cb9462 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:14.462137 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a08454-fdfd-477c-8f91-d25d23cb9462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:14.478259 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-73a08454-fdfd-477c-8f91-d25d23cb9462 None None] MFA Rules not processed for user `e548ea934bc14b7e921f7e161c5496cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:14.479706 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-73a08454-fdfd-477c-8f91-d25d23cb9462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:14.543899 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1160/2323] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:48:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:14.920169 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd14763-73ca-441f-b64d-0123d01a2da0 service cinder] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:14.932646 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd14763-73ca-441f-b64d-0123d01a2da0 service cinder] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:14.938874 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b408d48-6d7e-4574-adf9-ce64ba7f4aef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:14.939570 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b408d48-6d7e-4574-adf9-ce64ba7f4aef service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:14.940010 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b408d48-6d7e-4574-adf9-ce64ba7f4aef service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:14.940464 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b408d48-6d7e-4574-adf9-ce64ba7f4aef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:14.953581 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b408d48-6d7e-4574-adf9-ce64ba7f4aef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:14.955199 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b408d48-6d7e-4574-adf9-ce64ba7f4aef service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:14.967600 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1164/2324] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:48:14 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:48:20.056871 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a08454-fdfd-477c-8f91-d25d23cb9462 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:20.060269 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-73a08454-fdfd-477c-8f91-d25d23cb9462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:20.081024 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:20.177783 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a08454-fdfd-477c-8f91-d25d23cb9462 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:20.183985 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d1570d-60d6-4824-a99a-e38ec7256828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:20.185090 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d1570d-60d6-4824-a99a-e38ec7256828 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:20.185435 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d1570d-60d6-4824-a99a-e38ec7256828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:20.186283 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d1570d-60d6-4824-a99a-e38ec7256828 admin admin] PATH_INFO: `/v3/users/a1b2cdacd05f476ca8048238a69fa73d` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:20.187357 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d1570d-60d6-4824-a99a-e38ec7256828 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1b2cdacd05f476ca8048238a69fa73d)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:20.189523 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d1570d-60d6-4824-a99a-e38ec7256828 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:20.487725 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-12d1570d-60d6-4824-a99a-e38ec7256828 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1b2cdacd05f476ca8048238a69fa73d'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:20.596899 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-12d1570d-60d6-4824-a99a-e38ec7256828 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1b2cdacd05f476ca8048238a69fa73d'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:20.610973 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-12d1570d-60d6-4824-a99a-e38ec7256828 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1b2cdacd05f476ca8048238a69fa73d'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:20.625880 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-12d1570d-60d6-4824-a99a-e38ec7256828 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1b2cdacd05f476ca8048238a69fa73d'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:20.626701 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-12d1570d-60d6-4824-a99a-e38ec7256828 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1b2cdacd05f476ca8048238a69fa73d'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:20.636938 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1161/2325] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:48:20 2022] DELETE /identity/v3/users/a1b2cdacd05f476ca8048238a69fa73d => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:20.638880 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d93ae-6cba-4ac1-bdda-cdeab8842538 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:20.639382 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d93ae-6cba-4ac1-bdda-cdeab8842538 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:20.639478 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d93ae-6cba-4ac1-bdda-cdeab8842538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:20.661295 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-a21d93ae-6cba-4ac1-bdda-cdeab8842538 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:20.662523 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-a21d93ae-6cba-4ac1-bdda-cdeab8842538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:20.704634 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:20.739267 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1165/2326] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:48:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:20.750845 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d1570d-60d6-4824-a99a-e38ec7256828 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:20.752523 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-12d1570d-60d6-4824-a99a-e38ec7256828 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:20.815294 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d1570d-60d6-4824-a99a-e38ec7256828 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:20.819179 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9964d3b3-fa4f-49c2-aab0-b7b75d669b84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:20.819865 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9964d3b3-fa4f-49c2-aab0-b7b75d669b84 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:20.820171 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9964d3b3-fa4f-49c2-aab0-b7b75d669b84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:20.820463 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9964d3b3-fa4f-49c2-aab0-b7b75d669b84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:20.840983 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9964d3b3-fa4f-49c2-aab0-b7b75d669b84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:20.842676 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9964d3b3-fa4f-49c2-aab0-b7b75d669b84 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:20.853086 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1162/2327] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:48:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:48:21.586798 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21d93ae-6cba-4ac1-bdda-cdeab8842538 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:21.588263 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-a21d93ae-6cba-4ac1-bdda-cdeab8842538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:21.602266 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21d93ae-6cba-4ac1-bdda-cdeab8842538 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:21.608146 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4eab1a-fc82-4255-b175-bc5e8f3b02c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:21.611902 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4eab1a-fc82-4255-b175-bc5e8f3b02c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:21.611902 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4eab1a-fc82-4255-b175-bc5e8f3b02c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:21.611902 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4eab1a-fc82-4255-b175-bc5e8f3b02c1 admin admin] PATH_INFO: `/v3/projects/0b38e18678f445fd9b440e9f36af4888` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:21.611902 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4eab1a-fc82-4255-b175-bc5e8f3b02c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b38e18678f445fd9b440e9f36af4888)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:21.612785 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4eab1a-fc82-4255-b175-bc5e8f3b02c1 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:21.860160 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-7f4eab1a-fc82-4255-b175-bc5e8f3b02c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b38e18678f445fd9b440e9f36af4888', 'request_id': 'req-7f4eab1a-fc82-4255-b175-bc5e8f3b02c1'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:21.868376 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-7f4eab1a-fc82-4255-b175-bc5e8f3b02c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b38e18678f445fd9b440e9f36af4888', 'request_id': 'req-7f4eab1a-fc82-4255-b175-bc5e8f3b02c1'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:22.016956 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-7f4eab1a-fc82-4255-b175-bc5e8f3b02c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b38e18678f445fd9b440e9f36af4888', 'request_id': 'req-7f4eab1a-fc82-4255-b175-bc5e8f3b02c1'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:22.034279 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-7f4eab1a-fc82-4255-b175-bc5e8f3b02c1 admin admin] The token cache is being invalidate because project 0b38e18678f445fd9b440e9f36af4888 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:48:22.034654 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-7f4eab1a-fc82-4255-b175-bc5e8f3b02c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:22.035586 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-7f4eab1a-fc82-4255-b175-bc5e8f3b02c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:22.037620 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1166/2328] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:48:21 2022] DELETE /identity/v3/projects/0b38e18678f445fd9b440e9f36af4888 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:22.042258 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9964d3b3-fa4f-49c2-aab0-b7b75d669b84 service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:22.053406 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9964d3b3-fa4f-49c2-aab0-b7b75d669b84 service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:22.091849 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:22.118032 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4b9faa-7251-4b76-8b9e-9a0279854ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:22.119033 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4b9faa-7251-4b76-8b9e-9a0279854ff6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:22.119389 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4b9faa-7251-4b76-8b9e-9a0279854ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:22.119804 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4b9faa-7251-4b76-8b9e-9a0279854ff6 admin admin] PATH_INFO: `/v3/users/ce62854017b04fc7a1217e33f45962ee` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:22.121301 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4b9faa-7251-4b76-8b9e-9a0279854ff6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce62854017b04fc7a1217e33f45962ee)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:22.123327 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4b9faa-7251-4b76-8b9e-9a0279854ff6 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:22.507514 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-8b4b9faa-7251-4b76-8b9e-9a0279854ff6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce62854017b04fc7a1217e33f45962ee'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:22.670776 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-8b4b9faa-7251-4b76-8b9e-9a0279854ff6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce62854017b04fc7a1217e33f45962ee'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:22.678526 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-8b4b9faa-7251-4b76-8b9e-9a0279854ff6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce62854017b04fc7a1217e33f45962ee'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:22.691721 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-8b4b9faa-7251-4b76-8b9e-9a0279854ff6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce62854017b04fc7a1217e33f45962ee'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:22.692356 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-8b4b9faa-7251-4b76-8b9e-9a0279854ff6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce62854017b04fc7a1217e33f45962ee'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:22.698307 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1163/2329] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:48:22 2022] DELETE /identity/v3/users/ce62854017b04fc7a1217e33f45962ee => generated 0 bytes in 657 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:48:23.502882 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4eab1a-fc82-4255-b175-bc5e8f3b02c1 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:23.505845 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-7f4eab1a-fc82-4255-b175-bc5e8f3b02c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:23.530002 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4eab1a-fc82-4255-b175-bc5e8f3b02c1 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:23.601977 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d885c9c3-b70d-4719-af27-b3aaa240ec9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:23.603023 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d885c9c3-b70d-4719-af27-b3aaa240ec9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:23.603405 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d885c9c3-b70d-4719-af27-b3aaa240ec9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:23.610014 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d885c9c3-b70d-4719-af27-b3aaa240ec9b admin admin] PATH_INFO: `/v3/projects/1c7ab308903c4e7c843efe59d27d128f` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:23.610014 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d885c9c3-b70d-4719-af27-b3aaa240ec9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c7ab308903c4e7c843efe59d27d128f)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:23.610014 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d885c9c3-b70d-4719-af27-b3aaa240ec9b admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:23.917712 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-d885c9c3-b70d-4719-af27-b3aaa240ec9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c7ab308903c4e7c843efe59d27d128f', 'request_id': 'req-d885c9c3-b70d-4719-af27-b3aaa240ec9b'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:23.924393 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-d885c9c3-b70d-4719-af27-b3aaa240ec9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c7ab308903c4e7c843efe59d27d128f', 'request_id': 'req-d885c9c3-b70d-4719-af27-b3aaa240ec9b'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:24.014678 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-d885c9c3-b70d-4719-af27-b3aaa240ec9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c7ab308903c4e7c843efe59d27d128f', 'request_id': 'req-d885c9c3-b70d-4719-af27-b3aaa240ec9b'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:24.014678 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:24.024285 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-d885c9c3-b70d-4719-af27-b3aaa240ec9b admin admin] The token cache is being invalidate because project 1c7ab308903c4e7c843efe59d27d128f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:48:24.024576 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-d885c9c3-b70d-4719-af27-b3aaa240ec9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:24.025274 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-d885c9c3-b70d-4719-af27-b3aaa240ec9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:24.027516 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1167/2330] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:48:23 2022] DELETE /identity/v3/projects/1c7ab308903c4e7c843efe59d27d128f => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:24.032953 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86310a9-b36d-4994-8786-81d499f90386 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:24.033441 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86310a9-b36d-4994-8786-81d499f90386 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:24.033537 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86310a9-b36d-4994-8786-81d499f90386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:24.052861 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-b86310a9-b36d-4994-8786-81d499f90386 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:24.054081 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-b86310a9-b36d-4994-8786-81d499f90386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:24.113995 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d885c9c3-b70d-4719-af27-b3aaa240ec9b admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:24.116103 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-d885c9c3-b70d-4719-af27-b3aaa240ec9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:24.128428 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1164/2331] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:48:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:24.136608 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335273de-24c4-4072-9947-894432762fec None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:24.137035 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335273de-24c4-4072-9947-894432762fec None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:24.137523 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335273de-24c4-4072-9947-894432762fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:24.158559 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-335273de-24c4-4072-9947-894432762fec None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:24.159821 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-335273de-24c4-4072-9947-894432762fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:24.168150 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1165/2332] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:48:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:48:24.172556 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335273de-24c4-4072-9947-894432762fec None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:24.185182 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335273de-24c4-4072-9947-894432762fec None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:24.193607 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63301e4e-280a-4083-be08-e1ab720fd697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:24.194483 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63301e4e-280a-4083-be08-e1ab720fd697 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:24.194951 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63301e4e-280a-4083-be08-e1ab720fd697 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:24.195373 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63301e4e-280a-4083-be08-e1ab720fd697 admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:24.196279 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63301e4e-280a-4083-be08-e1ab720fd697 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:48:24.196748 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63301e4e-280a-4083-be08-e1ab720fd697 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:24.199361 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63301e4e-280a-4083-be08-e1ab720fd697 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:24.206082 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d885c9c3-b70d-4719-af27-b3aaa240ec9b admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:24.211608 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a6a02-950c-4e23-b346-25c61a12641c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:24.212511 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a6a02-950c-4e23-b346-25c61a12641c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:24.212801 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a6a02-950c-4e23-b346-25c61a12641c admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:24.213083 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a6a02-950c-4e23-b346-25c61a12641c admin admin] PATH_INFO: `/v3/users/04dea46a9cbc49afa81bc333f34c50db` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:24.215684 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5a6a02-950c-4e23-b346-25c61a12641c admin admin] RBAC: Authorizing `identity:delete_user(user_id=04dea46a9cbc49afa81bc333f34c50db)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:24.217414 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1166/2333] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 16:48:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:48:24.219898 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5a6a02-950c-4e23-b346-25c61a12641c admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:24.222650 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63301e4e-280a-4083-be08-e1ab720fd697 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:24.243212 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63301e4e-280a-4083-be08-e1ab720fd697 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:24.250278 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ba3c02-9c6e-4203-bb02-8d3c84ac5f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:24.251580 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba3c02-9c6e-4203-bb02-8d3c84ac5f3a admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:24.252155 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba3c02-9c6e-4203-bb02-8d3c84ac5f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:24.253562 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba3c02-9c6e-4203-bb02-8d3c84ac5f3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:24.254598 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ba3c02-9c6e-4203-bb02-8d3c84ac5f3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:24.257671 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ba3c02-9c6e-4203-bb02-8d3c84ac5f3a admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:24.351847 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1167/2334] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:48:24 2022] POST /identity/v3/projects => generated 381 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:48:24.351847 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4613d00-1ceb-479d-a8d8-9acec32200ac None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:24.351847 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4613d00-1ceb-479d-a8d8-9acec32200ac None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:24.351847 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4613d00-1ceb-479d-a8d8-9acec32200ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:24.370323 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-a4613d00-1ceb-479d-a8d8-9acec32200ac None None] MFA Rules not processed for user `642ba7eed36e4956834c06b771259a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:24.371746 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a4613d00-1ceb-479d-a8d8-9acec32200ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:24.380624 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:24.435244 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1168/2335] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:48:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:24.436255 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4613d00-1ceb-479d-a8d8-9acec32200ac None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:24.472252 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-0b5a6a02-950c-4e23-b346-25c61a12641c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04dea46a9cbc49afa81bc333f34c50db'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:24.509427 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4613d00-1ceb-479d-a8d8-9acec32200ac None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:24.524157 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9df39c4-6412-49f6-a2a3-95fbd6e24392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:24.528792 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df39c4-6412-49f6-a2a3-95fbd6e24392 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:24.528792 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df39c4-6412-49f6-a2a3-95fbd6e24392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:24.528792 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df39c4-6412-49f6-a2a3-95fbd6e24392 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:24.528792 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9df39c4-6412-49f6-a2a3-95fbd6e24392 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:24.528792 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9df39c4-6412-49f6-a2a3-95fbd6e24392 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:24.553588 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-0b5a6a02-950c-4e23-b346-25c61a12641c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04dea46a9cbc49afa81bc333f34c50db'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:24.554597 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-0b5a6a02-950c-4e23-b346-25c61a12641c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04dea46a9cbc49afa81bc333f34c50db'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:24.555376 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-0b5a6a02-950c-4e23-b346-25c61a12641c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04dea46a9cbc49afa81bc333f34c50db'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:24.568998 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-0b5a6a02-950c-4e23-b346-25c61a12641c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04dea46a9cbc49afa81bc333f34c50db'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:24.592833 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1168/2336] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:48:24 2022] DELETE /identity/v3/users/04dea46a9cbc49afa81bc333f34c50db => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:48:24.597451 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9243dc8-c2ce-4f0a-be76-2a81a25a423b None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:24.598549 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9243dc8-c2ce-4f0a-be76-2a81a25a423b None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:24.598549 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9243dc8-c2ce-4f0a-be76-2a81a25a423b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:24.616997 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-e9243dc8-c2ce-4f0a-be76-2a81a25a423b None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:24.618323 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-e9243dc8-c2ce-4f0a-be76-2a81a25a423b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:24.674811 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1169/2337] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:48:24 2022] POST /identity/v3/users => generated 291 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:48:24.676920 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9df39c4-6412-49f6-a2a3-95fbd6e24392 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:24.679688 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-b9df39c4-6412-49f6-a2a3-95fbd6e24392 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:24.688246 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1169/2338] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:48:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:24.697364 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9df39c4-6412-49f6-a2a3-95fbd6e24392 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:24.698968 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9243dc8-c2ce-4f0a-be76-2a81a25a423b None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:24.701910 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-e9243dc8-c2ce-4f0a-be76-2a81a25a423b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:24.706540 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f698381-92ad-4006-9365-fbc9bfefe12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:24.707231 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f698381-92ad-4006-9365-fbc9bfefe12d admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:24.707573 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f698381-92ad-4006-9365-fbc9bfefe12d admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:24.707825 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f698381-92ad-4006-9365-fbc9bfefe12d admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:24.708626 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f698381-92ad-4006-9365-fbc9bfefe12d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:24.710489 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f698381-92ad-4006-9365-fbc9bfefe12d admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:24.725684 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1170/2339] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:48:24 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:48:24.726673 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f698381-92ad-4006-9365-fbc9bfefe12d admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:24.738123 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f698381-92ad-4006-9365-fbc9bfefe12d admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:24.744093 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ba8e03-22d3-470a-86ca-3829925efdfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:24.745406 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ba8e03-22d3-470a-86ca-3829925efdfa admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:24.745824 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ba8e03-22d3-470a-86ca-3829925efdfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:24.746243 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ba8e03-22d3-470a-86ca-3829925efdfa admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:24.746984 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ba8e03-22d3-470a-86ca-3829925efdfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:24.748692 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ba8e03-22d3-470a-86ca-3829925efdfa admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:24.764660 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1171/2340] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:48:24 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:48:24.765479 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ba8e03-22d3-470a-86ca-3829925efdfa admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:24.777947 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ba8e03-22d3-470a-86ca-3829925efdfa admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:24.782502 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9243dc8-c2ce-4f0a-be76-2a81a25a423b None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:24.782862 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9b826b-ef3b-4f71-8830-870c655f4674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:24.783811 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b826b-ef3b-4f71-8830-870c655f4674 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:24.784100 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b826b-ef3b-4f71-8830-870c655f4674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:24.784374 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b826b-ef3b-4f71-8830-870c655f4674 admin admin] PATH_INFO: `/v3/projects/4c2ce510d4844e2aac010705f39c026f/users/d653dca40566437f8bba81a0450c4231/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:24.786575 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2d6fdd-79cf-4e1c-a047-3dcd3fe8a09b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:24.787249 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d6fdd-79cf-4e1c-a047-3dcd3fe8a09b service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:24.787562 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d6fdd-79cf-4e1c-a047-3dcd3fe8a09b service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:24.787844 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d6fdd-79cf-4e1c-a047-3dcd3fe8a09b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:24.802756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9b826b-ef3b-4f71-8830-870c655f4674 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c2ce510d4844e2aac010705f39c026f, user_id=d653dca40566437f8bba81a0450c4231, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:24.806446 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9b826b-ef3b-4f71-8830-870c655f4674 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:24.810311 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2d6fdd-79cf-4e1c-a047-3dcd3fe8a09b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:24.811555 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2d6fdd-79cf-4e1c-a047-3dcd3fe8a09b service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:24.823148 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1170/2341] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:48:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:48:24.889776 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:24.923648 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1172/2342] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:48:24 2022] PUT /identity/v3/projects/4c2ce510d4844e2aac010705f39c026f/users/d653dca40566437f8bba81a0450c4231/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:24.934673 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d842338e-ac32-4299-8546-d4dc798bc9aa None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:24.944134 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d842338e-ac32-4299-8546-d4dc798bc9aa None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:24.947665 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d842338e-ac32-4299-8546-d4dc798bc9aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:24.977009 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-d842338e-ac32-4299-8546-d4dc798bc9aa None None] MFA Rules not processed for user `d653dca40566437f8bba81a0450c4231`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:24.978484 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-d842338e-ac32-4299-8546-d4dc798bc9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:25.062012 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1171/2343] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:48:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:25.066296 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9b826b-ef3b-4f71-8830-870c655f4674 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:25.196126 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9b826b-ef3b-4f71-8830-870c655f4674 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:25.202846 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e3aa48-88fc-43b5-8601-a8a54107057c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:25.203579 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e3aa48-88fc-43b5-8601-a8a54107057c admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:25.203867 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e3aa48-88fc-43b5-8601-a8a54107057c admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:25.204145 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e3aa48-88fc-43b5-8601-a8a54107057c admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:25.204863 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e3aa48-88fc-43b5-8601-a8a54107057c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:25.207380 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e3aa48-88fc-43b5-8601-a8a54107057c admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:25.307973 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1173/2344] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:48:25 2022] POST /identity/v3/projects => generated 381 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:48:25.312293 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d842338e-ac32-4299-8546-d4dc798bc9aa None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:25.324089 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d842338e-ac32-4299-8546-d4dc798bc9aa None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:25.387174 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb368f5-e3d3-481e-9fc3-d7c833a3119f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:25.392027 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb368f5-e3d3-481e-9fc3-d7c833a3119f admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:25.392027 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb368f5-e3d3-481e-9fc3-d7c833a3119f admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:25.392027 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb368f5-e3d3-481e-9fc3-d7c833a3119f admin admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:25.392027 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb368f5-e3d3-481e-9fc3-d7c833a3119f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:25.392027 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb368f5-e3d3-481e-9fc3-d7c833a3119f admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:25.426029 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e3aa48-88fc-43b5-8601-a8a54107057c admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:25.428239 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-57e3aa48-88fc-43b5-8601-a8a54107057c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:25.443724 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e3aa48-88fc-43b5-8601-a8a54107057c admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:25.450834 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade2b8b-8dd5-4145-85e2-0d7ebfee2f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:25.452411 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade2b8b-8dd5-4145-85e2-0d7ebfee2f28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:25.452915 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade2b8b-8dd5-4145-85e2-0d7ebfee2f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:25.453262 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade2b8b-8dd5-4145-85e2-0d7ebfee2f28 admin admin] PATH_INFO: `/v3/projects/fcf58f51026c4ee2bb9d8d9452208dce` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:25.454611 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cade2b8b-8dd5-4145-85e2-0d7ebfee2f28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcf58f51026c4ee2bb9d8d9452208dce)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:25.456840 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cade2b8b-8dd5-4145-85e2-0d7ebfee2f28 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:25.497478 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1172/2345] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:48:25 2022] POST /identity/v3/users => generated 297 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:48:25.502963 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb368f5-e3d3-481e-9fc3-d7c833a3119f admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:25.522387 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb368f5-e3d3-481e-9fc3-d7c833a3119f admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:25.531049 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b104fe5a-866f-4b69-b7be-f8474e0a5ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:25.532304 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104fe5a-866f-4b69-b7be-f8474e0a5ed3 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:25.532462 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104fe5a-866f-4b69-b7be-f8474e0a5ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:25.532751 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104fe5a-866f-4b69-b7be-f8474e0a5ed3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:25.533746 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b104fe5a-866f-4b69-b7be-f8474e0a5ed3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:25.536813 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b104fe5a-866f-4b69-b7be-f8474e0a5ed3 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:25.556885 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1173/2346] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:48:25 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:48:25.564461 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b104fe5a-866f-4b69-b7be-f8474e0a5ed3 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:25.581335 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b104fe5a-866f-4b69-b7be-f8474e0a5ed3 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:25.593659 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-cade2b8b-8dd5-4145-85e2-0d7ebfee2f28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcf58f51026c4ee2bb9d8d9452208dce', 'request_id': 'req-cade2b8b-8dd5-4145-85e2-0d7ebfee2f28'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:25.608743 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-cade2b8b-8dd5-4145-85e2-0d7ebfee2f28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcf58f51026c4ee2bb9d8d9452208dce', 'request_id': 'req-cade2b8b-8dd5-4145-85e2-0d7ebfee2f28'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:25.647430 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-cade2b8b-8dd5-4145-85e2-0d7ebfee2f28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcf58f51026c4ee2bb9d8d9452208dce', 'request_id': 'req-cade2b8b-8dd5-4145-85e2-0d7ebfee2f28'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:25.698958 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080d4788-a4af-4431-be52-a1456d1f59b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:25.700142 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d4788-a4af-4431-be52-a1456d1f59b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:25.700500 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d4788-a4af-4431-be52-a1456d1f59b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:25.700936 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d4788-a4af-4431-be52-a1456d1f59b3 admin admin] PATH_INFO: `/v3/projects/e0c2aef15a934e41ab453d8d4e1cc4d4/users/1528d533652344ffa2460bd5e5d6a82e/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:25.719869 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080d4788-a4af-4431-be52-a1456d1f59b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0c2aef15a934e41ab453d8d4e1cc4d4, user_id=1528d533652344ffa2460bd5e5d6a82e, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:25.723105 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080d4788-a4af-4431-be52-a1456d1f59b3 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:25.742937 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-cade2b8b-8dd5-4145-85e2-0d7ebfee2f28 admin admin] The token cache is being invalidate because project fcf58f51026c4ee2bb9d8d9452208dce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:48:25.743361 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-cade2b8b-8dd5-4145-85e2-0d7ebfee2f28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:25.744651 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-cade2b8b-8dd5-4145-85e2-0d7ebfee2f28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:25.748292 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1174/2347] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:48:25 2022] DELETE /identity/v3/projects/fcf58f51026c4ee2bb9d8d9452208dce => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:48:25.753167 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975233dc-db9e-471f-b3e2-4198f998de08 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:25.753461 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975233dc-db9e-471f-b3e2-4198f998de08 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:25.753803 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975233dc-db9e-471f-b3e2-4198f998de08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:25.774746 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-975233dc-db9e-471f-b3e2-4198f998de08 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:25.777725 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-975233dc-db9e-471f-b3e2-4198f998de08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:25.787731 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1175/2348] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:48:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:48:25.796437 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd9f17-e9ac-49fb-8089-67cd1b8ce400 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:25.796766 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd9f17-e9ac-49fb-8089-67cd1b8ce400 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:25.797065 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd9f17-e9ac-49fb-8089-67cd1b8ce400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:25.810040 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1174/2349] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:48:25 2022] PUT /identity/v3/projects/e0c2aef15a934e41ab453d8d4e1cc4d4/users/1528d533652344ffa2460bd5e5d6a82e/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:25.814219 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080d4788-a4af-4431-be52-a1456d1f59b3 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:25.816008 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-080d4788-a4af-4431-be52-a1456d1f59b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:25.830703 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-21cd9f17-e9ac-49fb-8089-67cd1b8ce400 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:25.841034 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-21cd9f17-e9ac-49fb-8089-67cd1b8ce400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:25.865602 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:25.903063 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080d4788-a4af-4431-be52-a1456d1f59b3 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:25.915300 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d79e331-2be0-4804-99d1-093c6a59230e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:25.915300 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79e331-2be0-4804-99d1-093c6a59230e admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:25.915300 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79e331-2be0-4804-99d1-093c6a59230e admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:25.915300 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79e331-2be0-4804-99d1-093c6a59230e admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:25.915794 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d79e331-2be0-4804-99d1-093c6a59230e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:25.915794 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d79e331-2be0-4804-99d1-093c6a59230e admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:25.927667 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1175/2350] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:48:25 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:48:25.931731 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d79e331-2be0-4804-99d1-093c6a59230e admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:25.942229 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d79e331-2be0-4804-99d1-093c6a59230e admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:25.942407 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1176/2351] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:48:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:48:25.947082 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cd9f17-e9ac-49fb-8089-67cd1b8ce400 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:25.949442 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4382a646-2c7a-4e62-a080-7b430452f355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:25.950379 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382a646-2c7a-4e62-a080-7b430452f355 admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:25.950663 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382a646-2c7a-4e62-a080-7b430452f355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:25.950944 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382a646-2c7a-4e62-a080-7b430452f355 admin admin] PATH_INFO: `/v3/projects/e0c2aef15a934e41ab453d8d4e1cc4d4/users/1528d533652344ffa2460bd5e5d6a82e/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:25.953275 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4382a646-2c7a-4e62-a080-7b430452f355 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0c2aef15a934e41ab453d8d4e1cc4d4, user_id=1528d533652344ffa2460bd5e5d6a82e, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:25.955962 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4382a646-2c7a-4e62-a080-7b430452f355 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:25.957793 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cd9f17-e9ac-49fb-8089-67cd1b8ce400 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:25.965002 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213df312-339d-48ea-b06a-ed9e17e8612d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:25.965674 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213df312-339d-48ea-b06a-ed9e17e8612d admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:25.966177 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213df312-339d-48ea-b06a-ed9e17e8612d admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:25.967857 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213df312-339d-48ea-b06a-ed9e17e8612d admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:25.977677 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1176/2352] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:48:25 2022] PUT /identity/v3/projects/e0c2aef15a934e41ab453d8d4e1cc4d4/users/1528d533652344ffa2460bd5e5d6a82e/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:25.977876 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213df312-339d-48ea-b06a-ed9e17e8612d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:48:25.977876 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213df312-339d-48ea-b06a-ed9e17e8612d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:25.977876 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213df312-339d-48ea-b06a-ed9e17e8612d admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:25.983978 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a5dd5-a333-4f7b-8fee-cb14389133ef None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:25.984243 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a5dd5-a333-4f7b-8fee-cb14389133ef None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:25.988791 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1177/2353] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 16:48:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:48:25.988957 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a5dd5-a333-4f7b-8fee-cb14389133ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:25.990423 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213df312-339d-48ea-b06a-ed9e17e8612d admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:26.015090 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213df312-339d-48ea-b06a-ed9e17e8612d admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:26.050374 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-315a5dd5-a333-4f7b-8fee-cb14389133ef None None] MFA Rules not processed for user `1528d533652344ffa2460bd5e5d6a82e`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:26.058771 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-315a5dd5-a333-4f7b-8fee-cb14389133ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:26.107819 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338f37ed-af8e-43c7-927d-35204034fb84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:26.108540 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338f37ed-af8e-43c7-927d-35204034fb84 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:26.109011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338f37ed-af8e-43c7-927d-35204034fb84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:26.111701 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338f37ed-af8e-43c7-927d-35204034fb84 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:26.111701 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338f37ed-af8e-43c7-927d-35204034fb84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:26.112495 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338f37ed-af8e-43c7-927d-35204034fb84 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:26.142477 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1177/2354] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:48:25 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:26.155029 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a5dd5-a333-4f7b-8fee-cb14389133ef None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:26.156466 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-315a5dd5-a333-4f7b-8fee-cb14389133ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:26.218266 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a5dd5-a333-4f7b-8fee-cb14389133ef None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:26.222145 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92011f8-9af3-48e7-9a32-8583e87680db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:26.222821 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92011f8-9af3-48e7-9a32-8583e87680db service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:26.223100 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92011f8-9af3-48e7-9a32-8583e87680db service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:26.223361 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92011f8-9af3-48e7-9a32-8583e87680db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:26.233410 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92011f8-9af3-48e7-9a32-8583e87680db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:26.235407 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92011f8-9af3-48e7-9a32-8583e87680db service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:26.245879 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1178/2355] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:48:26 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:48:26.267619 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1178/2356] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:48:25 2022] POST /identity/v3/projects => generated 383 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:48:26.272936 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92011f8-9af3-48e7-9a32-8583e87680db service cinder] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:26.283055 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92011f8-9af3-48e7-9a32-8583e87680db service cinder] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:26.345064 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9453768-49c3-446b-9ecb-57d00d0d5246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:26.345743 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9453768-49c3-446b-9ecb-57d00d0d5246 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:26.346027 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9453768-49c3-446b-9ecb-57d00d0d5246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:26.346346 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9453768-49c3-446b-9ecb-57d00d0d5246 admin admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:26.347018 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9453768-49c3-446b-9ecb-57d00d0d5246 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:26.348986 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9453768-49c3-446b-9ecb-57d00d0d5246 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:26.455309 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338f37ed-af8e-43c7-927d-35204034fb84 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:26.457237 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-338f37ed-af8e-43c7-927d-35204034fb84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:26.517989 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338f37ed-af8e-43c7-927d-35204034fb84 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:26.522233 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d235fdc2-e219-4592-a1d1-d7e10a9e6225 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:26.522918 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235fdc2-e219-4592-a1d1-d7e10a9e6225 service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:26.523225 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235fdc2-e219-4592-a1d1-d7e10a9e6225 service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:26.523482 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235fdc2-e219-4592-a1d1-d7e10a9e6225 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:26.525693 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d235fdc2-e219-4592-a1d1-d7e10a9e6225 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:26.552468 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1179/2357] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:48:26 2022] POST /identity/v3/users => generated 292 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:48:26.555325 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9453768-49c3-446b-9ecb-57d00d0d5246 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:26.570604 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9453768-49c3-446b-9ecb-57d00d0d5246 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:26.577015 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d348b70-dee7-4341-8350-0e6a5f3c43fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:26.577971 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d348b70-dee7-4341-8350-0e6a5f3c43fa admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:26.578258 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d348b70-dee7-4341-8350-0e6a5f3c43fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:26.578538 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d348b70-dee7-4341-8350-0e6a5f3c43fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:26.579283 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d348b70-dee7-4341-8350-0e6a5f3c43fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:26.581073 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d348b70-dee7-4341-8350-0e6a5f3c43fa admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:26.596993 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1180/2358] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:48:26 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:48:26.597939 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d348b70-dee7-4341-8350-0e6a5f3c43fa admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:26.601289 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d235fdc2-e219-4592-a1d1-d7e10a9e6225 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:26.602402 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d235fdc2-e219-4592-a1d1-d7e10a9e6225 service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:26.620438 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d348b70-dee7-4341-8350-0e6a5f3c43fa admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:26.620815 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1179/2359] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:48:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:48:26.620914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fa8c37-5f9e-4305-af2e-9978626340b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:26.620914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa8c37-5f9e-4305-af2e-9978626340b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:26.620914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa8c37-5f9e-4305-af2e-9978626340b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:26.620914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa8c37-5f9e-4305-af2e-9978626340b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:26.621254 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fa8c37-5f9e-4305-af2e-9978626340b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:26.621254 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fa8c37-5f9e-4305-af2e-9978626340b0 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:26.632379 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1181/2360] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:48:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:48:26.639016 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d235fdc2-e219-4592-a1d1-d7e10a9e6225 service cinder] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:26.650284 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d235fdc2-e219-4592-a1d1-d7e10a9e6225 service cinder] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:26.656621 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7e852b-1551-4005-a9ce-0dd814d2a578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:26.657516 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e852b-1551-4005-a9ce-0dd814d2a578 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:26.657842 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e852b-1551-4005-a9ce-0dd814d2a578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:26.658167 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e852b-1551-4005-a9ce-0dd814d2a578 admin admin] PATH_INFO: `/v3/projects/8368f31817254dafaea912485f3aa86b/users/a9452b2e7f334f0387ecfe7194a7b38b/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:26.680221 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7e852b-1551-4005-a9ce-0dd814d2a578 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8368f31817254dafaea912485f3aa86b, user_id=a9452b2e7f334f0387ecfe7194a7b38b, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:26.683553 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7e852b-1551-4005-a9ce-0dd814d2a578 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:26.743598 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1180/2361] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:48:26 2022] PUT /identity/v3/projects/8368f31817254dafaea912485f3aa86b/users/a9452b2e7f334f0387ecfe7194a7b38b/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:26.749326 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c74ef6-8785-4917-b914-f06274634ec2 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:26.749623 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c74ef6-8785-4917-b914-f06274634ec2 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:26.749930 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c74ef6-8785-4917-b914-f06274634ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:26.769250 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-d7c74ef6-8785-4917-b914-f06274634ec2 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:26.773604 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-d7c74ef6-8785-4917-b914-f06274634ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:26.905313 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1182/2362] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:48:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:26.915290 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7e852b-1551-4005-a9ce-0dd814d2a578 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:26.918166 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-9c7e852b-1551-4005-a9ce-0dd814d2a578 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:26.974091 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:26.990720 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7e852b-1551-4005-a9ce-0dd814d2a578 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:26.995961 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214325f3-d5f4-4b64-b2ea-98762a0b5048 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:26.996633 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214325f3-d5f4-4b64-b2ea-98762a0b5048 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:26.996920 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214325f3-d5f4-4b64-b2ea-98762a0b5048 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:26.997296 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214325f3-d5f4-4b64-b2ea-98762a0b5048 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:27.011367 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214325f3-d5f4-4b64-b2ea-98762a0b5048 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:27.012969 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214325f3-d5f4-4b64-b2ea-98762a0b5048 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:27.023520 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1181/2363] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:48:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:48:27.180304 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c74ef6-8785-4917-b914-f06274634ec2 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:27.194719 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c74ef6-8785-4917-b914-f06274634ec2 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:27.204778 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfda6a9-0faa-4a73-a7a3-5328ec804fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:27.205908 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfda6a9-0faa-4a73-a7a3-5328ec804fdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:27.206247 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfda6a9-0faa-4a73-a7a3-5328ec804fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:27.206531 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfda6a9-0faa-4a73-a7a3-5328ec804fdb admin admin] PATH_INFO: `/v3/users/d653dca40566437f8bba81a0450c4231` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:27.208934 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfda6a9-0faa-4a73-a7a3-5328ec804fdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=d653dca40566437f8bba81a0450c4231)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:27.214470 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfda6a9-0faa-4a73-a7a3-5328ec804fdb admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:27.505299 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-edfda6a9-0faa-4a73-a7a3-5328ec804fdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd653dca40566437f8bba81a0450c4231'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:27.589510 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-edfda6a9-0faa-4a73-a7a3-5328ec804fdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd653dca40566437f8bba81a0450c4231'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:27.590369 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-edfda6a9-0faa-4a73-a7a3-5328ec804fdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd653dca40566437f8bba81a0450c4231'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:27.591454 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-edfda6a9-0faa-4a73-a7a3-5328ec804fdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd653dca40566437f8bba81a0450c4231'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:27.601854 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-edfda6a9-0faa-4a73-a7a3-5328ec804fdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd653dca40566437f8bba81a0450c4231'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:27.633347 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1183/2364] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:48:27 2022] DELETE /identity/v3/users/d653dca40566437f8bba81a0450c4231 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:27.640036 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a35df56-65bf-49bd-b66d-367df29740af None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:27.640884 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a35df56-65bf-49bd-b66d-367df29740af None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:27.640884 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a35df56-65bf-49bd-b66d-367df29740af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:27.658718 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-7a35df56-65bf-49bd-b66d-367df29740af None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:27.659967 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-7a35df56-65bf-49bd-b66d-367df29740af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:27.725247 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1182/2365] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:48:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:48:27.739511 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfda6a9-0faa-4a73-a7a3-5328ec804fdb admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:27.741082 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-edfda6a9-0faa-4a73-a7a3-5328ec804fdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:27.806923 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfda6a9-0faa-4a73-a7a3-5328ec804fdb admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:27.812211 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026b336-d338-458c-a37b-20fd9bfa1ded service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:27.813153 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026b336-d338-458c-a37b-20fd9bfa1ded service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:27.813747 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026b336-d338-458c-a37b-20fd9bfa1ded service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:27.814001 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026b336-d338-458c-a37b-20fd9bfa1ded service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:27.827014 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3026b336-d338-458c-a37b-20fd9bfa1ded service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:27.828793 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3026b336-d338-458c-a37b-20fd9bfa1ded service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:27.843829 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1184/2366] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:48:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:48:27.991332 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:28.781596 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a35df56-65bf-49bd-b66d-367df29740af None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:28.783147 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-7a35df56-65bf-49bd-b66d-367df29740af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:28.804552 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a35df56-65bf-49bd-b66d-367df29740af None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:28.810415 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45b0b47-7eef-4561-9d64-269ccea66f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:28.811635 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45b0b47-7eef-4561-9d64-269ccea66f78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:28.812122 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45b0b47-7eef-4561-9d64-269ccea66f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:28.812638 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45b0b47-7eef-4561-9d64-269ccea66f78 admin admin] PATH_INFO: `/v3/projects/4c2ce510d4844e2aac010705f39c026f` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:28.814199 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45b0b47-7eef-4561-9d64-269ccea66f78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c2ce510d4844e2aac010705f39c026f)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:28.822533 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45b0b47-7eef-4561-9d64-269ccea66f78 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:28.882559 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a836d0-0ae3-490e-9367-81a63bcd9072 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:28.883063 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a836d0-0ae3-490e-9367-81a63bcd9072 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:28.883156 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a836d0-0ae3-490e-9367-81a63bcd9072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:28.915621 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-68a836d0-0ae3-490e-9367-81a63bcd9072 None None] MFA Rules not processed for user `a9452b2e7f334f0387ecfe7194a7b38b`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:28.916897 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-68a836d0-0ae3-490e-9367-81a63bcd9072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:28.950050 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-d45b0b47-7eef-4561-9d64-269ccea66f78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c2ce510d4844e2aac010705f39c026f', 'request_id': 'req-d45b0b47-7eef-4561-9d64-269ccea66f78'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:28.962478 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-d45b0b47-7eef-4561-9d64-269ccea66f78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c2ce510d4844e2aac010705f39c026f', 'request_id': 'req-d45b0b47-7eef-4561-9d64-269ccea66f78'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:28.973037 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-d45b0b47-7eef-4561-9d64-269ccea66f78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c2ce510d4844e2aac010705f39c026f', 'request_id': 'req-d45b0b47-7eef-4561-9d64-269ccea66f78'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:29.039287 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1185/2367] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:48:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:48:29.052404 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a836d0-0ae3-490e-9367-81a63bcd9072 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:29.053926 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-68a836d0-0ae3-490e-9367-81a63bcd9072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:29.126595 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-d45b0b47-7eef-4561-9d64-269ccea66f78 admin admin] The token cache is being invalidate because project 4c2ce510d4844e2aac010705f39c026f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:48:29.127175 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-d45b0b47-7eef-4561-9d64-269ccea66f78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:29.128698 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-d45b0b47-7eef-4561-9d64-269ccea66f78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:29.130573 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1183/2368] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:48:28 2022] DELETE /identity/v3/projects/4c2ce510d4844e2aac010705f39c026f => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:29.134542 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45b0b47-7eef-4561-9d64-269ccea66f78 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:29.136629 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a836d0-0ae3-490e-9367-81a63bcd9072 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:29.138067 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d45b0b47-7eef-4561-9d64-269ccea66f78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:29.212874 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f573ad2-5d77-4f29-b850-45e4f48ea518 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:29.213210 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f573ad2-5d77-4f29-b850-45e4f48ea518 service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:29.213620 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f573ad2-5d77-4f29-b850-45e4f48ea518 service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:29.214699 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f573ad2-5d77-4f29-b850-45e4f48ea518 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:29.216972 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-1f573ad2-5d77-4f29-b850-45e4f48ea518 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:29.220133 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45b0b47-7eef-4561-9d64-269ccea66f78 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:29.225351 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa9bbb1-880e-4308-bd44-6f5ac6cd9244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:29.226435 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9bbb1-880e-4308-bd44-6f5ac6cd9244 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:29.226737 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9bbb1-880e-4308-bd44-6f5ac6cd9244 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:29.227225 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9bbb1-880e-4308-bd44-6f5ac6cd9244 admin admin] PATH_INFO: `/v3/users/1528d533652344ffa2460bd5e5d6a82e` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:29.228796 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa9bbb1-880e-4308-bd44-6f5ac6cd9244 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1528d533652344ffa2460bd5e5d6a82e)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:29.231104 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa9bbb1-880e-4308-bd44-6f5ac6cd9244 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:29.315556 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f573ad2-5d77-4f29-b850-45e4f48ea518 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:29.317070 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f573ad2-5d77-4f29-b850-45e4f48ea518 service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:29.329794 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1186/2369] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:48:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:48:29.529627 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-5fa9bbb1-880e-4308-bd44-6f5ac6cd9244 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1528d533652344ffa2460bd5e5d6a82e'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:29.642377 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-5fa9bbb1-880e-4308-bd44-6f5ac6cd9244 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1528d533652344ffa2460bd5e5d6a82e'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:29.644539 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:29.651362 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-5fa9bbb1-880e-4308-bd44-6f5ac6cd9244 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1528d533652344ffa2460bd5e5d6a82e'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:29.664309 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-5fa9bbb1-880e-4308-bd44-6f5ac6cd9244 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1528d533652344ffa2460bd5e5d6a82e'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:29.665159 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-5fa9bbb1-880e-4308-bd44-6f5ac6cd9244 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1528d533652344ffa2460bd5e5d6a82e'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:29.672085 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1184/2370] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:48:29 2022] DELETE /identity/v3/users/1528d533652344ffa2460bd5e5d6a82e => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:30.331694 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f573ad2-5d77-4f29-b850-45e4f48ea518 service cinder] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:30.349281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f573ad2-5d77-4f29-b850-45e4f48ea518 service cinder] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:30.667900 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bedd58e-04db-4ca6-a2c8-fa64d6aeb9c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:30.668827 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bedd58e-04db-4ca6-a2c8-fa64d6aeb9c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:30.669112 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bedd58e-04db-4ca6-a2c8-fa64d6aeb9c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:30.669384 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bedd58e-04db-4ca6-a2c8-fa64d6aeb9c9 admin admin] PATH_INFO: `/v3/projects/e0c2aef15a934e41ab453d8d4e1cc4d4` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:30.671329 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bedd58e-04db-4ca6-a2c8-fa64d6aeb9c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0c2aef15a934e41ab453d8d4e1cc4d4)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:30.673730 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bedd58e-04db-4ca6-a2c8-fa64d6aeb9c9 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:30.812675 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-9bedd58e-04db-4ca6-a2c8-fa64d6aeb9c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0c2aef15a934e41ab453d8d4e1cc4d4', 'request_id': 'req-9bedd58e-04db-4ca6-a2c8-fa64d6aeb9c9'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:30.819196 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-9bedd58e-04db-4ca6-a2c8-fa64d6aeb9c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0c2aef15a934e41ab453d8d4e1cc4d4', 'request_id': 'req-9bedd58e-04db-4ca6-a2c8-fa64d6aeb9c9'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:30.932521 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-9bedd58e-04db-4ca6-a2c8-fa64d6aeb9c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0c2aef15a934e41ab453d8d4e1cc4d4', 'request_id': 'req-9bedd58e-04db-4ca6-a2c8-fa64d6aeb9c9'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:30.947500 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-9bedd58e-04db-4ca6-a2c8-fa64d6aeb9c9 admin admin] The token cache is being invalidate because project e0c2aef15a934e41ab453d8d4e1cc4d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:48:30.948754 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-9bedd58e-04db-4ca6-a2c8-fa64d6aeb9c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:30.949566 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-9bedd58e-04db-4ca6-a2c8-fa64d6aeb9c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:30.951447 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1187/2371] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:48:30 2022] DELETE /identity/v3/projects/e0c2aef15a934e41ab453d8d4e1cc4d4 => generated 0 bytes in 620 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:30.956891 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab9a3c-bc06-4cfc-a8d2-9e7d31ef4105 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:30.957386 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab9a3c-bc06-4cfc-a8d2-9e7d31ef4105 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:30.957475 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab9a3c-bc06-4cfc-a8d2-9e7d31ef4105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:30.980959 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-8bab9a3c-bc06-4cfc-a8d2-9e7d31ef4105 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:30.982249 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-8bab9a3c-bc06-4cfc-a8d2-9e7d31ef4105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:31.089924 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1185/2372] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:48:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:31.098382 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d71c03-2ed2-451c-83bc-84f96905e742 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:31.098733 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d71c03-2ed2-451c-83bc-84f96905e742 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:31.099026 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d71c03-2ed2-451c-83bc-84f96905e742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:31.124901 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-78d71c03-2ed2-451c-83bc-84f96905e742 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:31.124901 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-78d71c03-2ed2-451c-83bc-84f96905e742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:31.130931 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1188/2373] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:48:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:31.136303 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bab9a3c-bc06-4cfc-a8d2-9e7d31ef4105 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:31.146397 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bab9a3c-bc06-4cfc-a8d2-9e7d31ef4105 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:31.151682 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409be0f6-148a-40f7-9540-11a6704c6ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:31.152378 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409be0f6-148a-40f7-9540-11a6704c6ec3 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:31.152948 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409be0f6-148a-40f7-9540-11a6704c6ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:31.153234 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409be0f6-148a-40f7-9540-11a6704c6ec3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:31.154049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409be0f6-148a-40f7-9540-11a6704c6ec3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:48:31.154362 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409be0f6-148a-40f7-9540-11a6704c6ec3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:31.156075 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409be0f6-148a-40f7-9540-11a6704c6ec3 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:31.173673 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1186/2374] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 16:48:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:48:31.178994 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d71c03-2ed2-451c-83bc-84f96905e742 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:31.195761 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d71c03-2ed2-451c-83bc-84f96905e742 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:31.215689 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9cf20e-bf3d-42e8-a5f1-f651dd0f29f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:31.216472 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9cf20e-bf3d-42e8-a5f1-f651dd0f29f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:31.216943 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9cf20e-bf3d-42e8-a5f1-f651dd0f29f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:31.217310 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9cf20e-bf3d-42e8-a5f1-f651dd0f29f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:31.218052 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9cf20e-bf3d-42e8-a5f1-f651dd0f29f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:31.220382 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9cf20e-bf3d-42e8-a5f1-f651dd0f29f3 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:31.451661 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f2e488-3473-4c3b-b649-39e02a6fdf23 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:31.452190 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f2e488-3473-4c3b-b649-39e02a6fdf23 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:31.452856 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f2e488-3473-4c3b-b649-39e02a6fdf23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:31.468714 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1189/2375] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:48:31 2022] POST /identity/v3/projects => generated 383 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:48:31.472661 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9cf20e-bf3d-42e8-a5f1-f651dd0f29f3 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:31.502296 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-05f2e488-3473-4c3b-b649-39e02a6fdf23 None None] MFA Rules not processed for user `a9452b2e7f334f0387ecfe7194a7b38b`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:31.504367 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-05f2e488-3473-4c3b-b649-39e02a6fdf23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:31.619668 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1187/2376] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:48:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:48:31.621730 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9cf20e-bf3d-42e8-a5f1-f651dd0f29f3 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:31.626724 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7f95fc-5af6-46cb-949d-8d2545e36fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:31.627426 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f95fc-5af6-46cb-949d-8d2545e36fda admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:31.627747 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f95fc-5af6-46cb-949d-8d2545e36fda admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:31.628031 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f95fc-5af6-46cb-949d-8d2545e36fda admin admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:31.628795 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7f95fc-5af6-46cb-949d-8d2545e36fda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:31.631983 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7f95fc-5af6-46cb-949d-8d2545e36fda admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:31.794850 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1190/2377] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:48:31 2022] POST /identity/v3/users => generated 292 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:48:31.796919 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f2e488-3473-4c3b-b649-39e02a6fdf23 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:31.815962 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f2e488-3473-4c3b-b649-39e02a6fdf23 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:31.825832 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9943fe3-54b5-4d37-b501-0c5eb85776ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:31.827095 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9943fe3-54b5-4d37-b501-0c5eb85776ce admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:31.840462 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9943fe3-54b5-4d37-b501-0c5eb85776ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:31.840462 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9943fe3-54b5-4d37-b501-0c5eb85776ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:31.842036 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9943fe3-54b5-4d37-b501-0c5eb85776ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:31.843003 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9943fe3-54b5-4d37-b501-0c5eb85776ce admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:31.854064 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1188/2378] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:48:31 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:48:31.858460 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7f95fc-5af6-46cb-949d-8d2545e36fda admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:31.873019 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7f95fc-5af6-46cb-949d-8d2545e36fda admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:31.878079 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d5c6a8-5376-47ea-9189-f945e5f7cb1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:31.880538 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5c6a8-5376-47ea-9189-f945e5f7cb1e admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:31.880538 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5c6a8-5376-47ea-9189-f945e5f7cb1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:31.880538 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5c6a8-5376-47ea-9189-f945e5f7cb1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:31.880538 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d5c6a8-5376-47ea-9189-f945e5f7cb1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:31.881861 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d5c6a8-5376-47ea-9189-f945e5f7cb1e admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:31.896831 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1191/2379] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:48:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:48:31.898225 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9943fe3-54b5-4d37-b501-0c5eb85776ce admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:31.909133 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9943fe3-54b5-4d37-b501-0c5eb85776ce admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:31.914674 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2af718-e9c2-4177-95d3-63644bbac61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:31.915626 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2af718-e9c2-4177-95d3-63644bbac61d admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:31.915912 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2af718-e9c2-4177-95d3-63644bbac61d admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:31.916319 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2af718-e9c2-4177-95d3-63644bbac61d admin admin] PATH_INFO: `/v3/projects/45099f0404f5499b9aaa76c3e79a65b6/users/2c97711bd4244faa8a37bd669af2c369/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:31.934618 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2af718-e9c2-4177-95d3-63644bbac61d admin admin] RBAC: Authorizing `identity:create_grant(project_id=45099f0404f5499b9aaa76c3e79a65b6, user_id=2c97711bd4244faa8a37bd669af2c369, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:31.937631 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2af718-e9c2-4177-95d3-63644bbac61d admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:32.062957 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1189/2380] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:48:31 2022] PUT /identity/v3/projects/45099f0404f5499b9aaa76c3e79a65b6/users/2c97711bd4244faa8a37bd669af2c369/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:32.076851 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd44c33-5f5c-412c-a957-e6d4890569f4 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:32.077160 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd44c33-5f5c-412c-a957-e6d4890569f4 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:32.077439 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd44c33-5f5c-412c-a957-e6d4890569f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:32.112204 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-5cd44c33-5f5c-412c-a957-e6d4890569f4 None None] MFA Rules not processed for user `2c97711bd4244faa8a37bd669af2c369`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:32.113654 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-5cd44c33-5f5c-412c-a957-e6d4890569f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:32.192683 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1192/2381] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:48:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:32.197379 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2af718-e9c2-4177-95d3-63644bbac61d admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:32.263430 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2af718-e9c2-4177-95d3-63644bbac61d admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:32.268295 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9405c6-2d6e-41e0-906e-222a0293d80b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:32.268956 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9405c6-2d6e-41e0-906e-222a0293d80b admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:32.269243 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9405c6-2d6e-41e0-906e-222a0293d80b admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:32.269600 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9405c6-2d6e-41e0-906e-222a0293d80b admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:32.270311 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9405c6-2d6e-41e0-906e-222a0293d80b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:32.272274 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9405c6-2d6e-41e0-906e-222a0293d80b admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:32.342804 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:32.410436 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1190/2382] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:48:32 2022] POST /identity/v3/projects => generated 383 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:48:32.414597 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd44c33-5f5c-412c-a957-e6d4890569f4 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:32.427413 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd44c33-5f5c-412c-a957-e6d4890569f4 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:32.498659 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f96653-81d1-426e-8bad-2b8fb04c313b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:32.499379 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f96653-81d1-426e-8bad-2b8fb04c313b admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:32.499663 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f96653-81d1-426e-8bad-2b8fb04c313b admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:32.499940 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f96653-81d1-426e-8bad-2b8fb04c313b admin admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:32.500677 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f96653-81d1-426e-8bad-2b8fb04c313b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:32.503225 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f96653-81d1-426e-8bad-2b8fb04c313b admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:32.594941 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1193/2383] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:48:32 2022] POST /identity/v3/users => generated 298 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:48:32.600039 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9405c6-2d6e-41e0-906e-222a0293d80b admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:32.612671 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9405c6-2d6e-41e0-906e-222a0293d80b admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:32.614609 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:32.618506 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0784b616-b8c6-4ad4-a05e-8e839430fde6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:32.619257 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0784b616-b8c6-4ad4-a05e-8e839430fde6 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:32.619656 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0784b616-b8c6-4ad4-a05e-8e839430fde6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:32.620020 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0784b616-b8c6-4ad4-a05e-8e839430fde6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:32.620841 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0784b616-b8c6-4ad4-a05e-8e839430fde6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:32.622531 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0784b616-b8c6-4ad4-a05e-8e839430fde6 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:32.635598 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1191/2384] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:48:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:48:32.640257 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f96653-81d1-426e-8bad-2b8fb04c313b admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:32.651927 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f96653-81d1-426e-8bad-2b8fb04c313b admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:32.657205 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c1aaf-03ed-46dd-9a92-68274cce2e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:32.658125 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c1aaf-03ed-46dd-9a92-68274cce2e40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:32.658437 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c1aaf-03ed-46dd-9a92-68274cce2e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:32.658713 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c1aaf-03ed-46dd-9a92-68274cce2e40 admin admin] PATH_INFO: `/v3/projects/cbc2045be1b84df6889fd32e0f134eb7/users/3419c2d629e944b6b7cd738276181795/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:32.675022 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08c1aaf-03ed-46dd-9a92-68274cce2e40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbc2045be1b84df6889fd32e0f134eb7, user_id=3419c2d629e944b6b7cd738276181795, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:32.677757 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08c1aaf-03ed-46dd-9a92-68274cce2e40 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:32.789817 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1194/2385] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:48:32 2022] PUT /identity/v3/projects/cbc2045be1b84df6889fd32e0f134eb7/users/3419c2d629e944b6b7cd738276181795/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:32.790987 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0784b616-b8c6-4ad4-a05e-8e839430fde6 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:32.802126 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0784b616-b8c6-4ad4-a05e-8e839430fde6 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:32.888857 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb69fbfe-d5fc-43dd-915e-a882694796b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:32.889583 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb69fbfe-d5fc-43dd-915e-a882694796b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:32.889916 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb69fbfe-d5fc-43dd-915e-a882694796b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:32.891156 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb69fbfe-d5fc-43dd-915e-a882694796b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:32.892205 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb69fbfe-d5fc-43dd-915e-a882694796b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:32.895993 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb69fbfe-d5fc-43dd-915e-a882694796b7 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:32.909326 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1192/2386] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:48:32 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:48:32.912105 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c1aaf-03ed-46dd-9a92-68274cce2e40 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:32.924983 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c1aaf-03ed-46dd-9a92-68274cce2e40 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:32.940917 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f7002c-e0a5-4eba-96c0-b2e3d19024d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:32.942027 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7002c-e0a5-4eba-96c0-b2e3d19024d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:32.942414 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7002c-e0a5-4eba-96c0-b2e3d19024d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:32.942763 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7002c-e0a5-4eba-96c0-b2e3d19024d9 admin admin] PATH_INFO: `/v3/projects/cbc2045be1b84df6889fd32e0f134eb7/users/3419c2d629e944b6b7cd738276181795/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:32.945339 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f7002c-e0a5-4eba-96c0-b2e3d19024d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbc2045be1b84df6889fd32e0f134eb7, user_id=3419c2d629e944b6b7cd738276181795, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:32.953122 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f7002c-e0a5-4eba-96c0-b2e3d19024d9 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:32.976329 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1195/2387] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:48:32 2022] PUT /identity/v3/projects/cbc2045be1b84df6889fd32e0f134eb7/users/3419c2d629e944b6b7cd738276181795/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:32.986991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b37ff4e-4fbd-4f11-9663-271cdc79e683 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:32.987289 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b37ff4e-4fbd-4f11-9663-271cdc79e683 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:32.987608 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b37ff4e-4fbd-4f11-9663-271cdc79e683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:33.057673 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-7b37ff4e-4fbd-4f11-9663-271cdc79e683 None None] MFA Rules not processed for user `3419c2d629e944b6b7cd738276181795`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:33.057673 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-7b37ff4e-4fbd-4f11-9663-271cdc79e683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:33.168510 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1193/2388] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:48:32 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:33.184230 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f7002c-e0a5-4eba-96c0-b2e3d19024d9 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:33.186401 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-a4f7002c-e0a5-4eba-96c0-b2e3d19024d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:33.297384 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f7002c-e0a5-4eba-96c0-b2e3d19024d9 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:33.302267 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0c1fbc-220c-4ad2-9edb-a70221652d56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:33.303279 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c1fbc-220c-4ad2-9edb-a70221652d56 service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:33.303593 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c1fbc-220c-4ad2-9edb-a70221652d56 service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:33.305239 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c1fbc-220c-4ad2-9edb-a70221652d56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:33.415360 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0c1fbc-220c-4ad2-9edb-a70221652d56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:33.418467 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0c1fbc-220c-4ad2-9edb-a70221652d56 service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:33.455269 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1196/2389] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:48:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:48:33.788769 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b37ff4e-4fbd-4f11-9663-271cdc79e683 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:33.794140 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-7b37ff4e-4fbd-4f11-9663-271cdc79e683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:33.879682 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b37ff4e-4fbd-4f11-9663-271cdc79e683 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:33.883378 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893a5727-3161-41f4-9735-0b9671d14c4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:33.886635 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a5727-3161-41f4-9735-0b9671d14c4a service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:33.887317 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a5727-3161-41f4-9735-0b9671d14c4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:33.887731 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a5727-3161-41f4-9735-0b9671d14c4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:33.890544 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-893a5727-3161-41f4-9735-0b9671d14c4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:33.959645 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893a5727-3161-41f4-9735-0b9671d14c4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:33.961147 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893a5727-3161-41f4-9735-0b9671d14c4a service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:33.972004 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1194/2390] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:48:33 2022] GET /identity/v3/auth/tokens => generated 3295 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:48:34.138255 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:35.283697 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:36.857663 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0c1fbc-220c-4ad2-9edb-a70221652d56 service cinder] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:36.859234 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-0a0c1fbc-220c-4ad2-9edb-a70221652d56 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:36.960154 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0c1fbc-220c-4ad2-9edb-a70221652d56 service cinder] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:36.965642 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77bbd8f-5c07-45c1-b6f4-5dc2ad439971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:36.967030 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77bbd8f-5c07-45c1-b6f4-5dc2ad439971 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:36.967339 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77bbd8f-5c07-45c1-b6f4-5dc2ad439971 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:36.967661 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77bbd8f-5c07-45c1-b6f4-5dc2ad439971 admin admin] PATH_INFO: `/v3/users/ea704b4947e8477cb261f3a61c786155` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:36.969150 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77bbd8f-5c07-45c1-b6f4-5dc2ad439971 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea704b4947e8477cb261f3a61c786155)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:36.972024 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77bbd8f-5c07-45c1-b6f4-5dc2ad439971 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:37.020877 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:37.191970 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-a77bbd8f-5c07-45c1-b6f4-5dc2ad439971 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea704b4947e8477cb261f3a61c786155'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:37.269084 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-a77bbd8f-5c07-45c1-b6f4-5dc2ad439971 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea704b4947e8477cb261f3a61c786155'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:37.274290 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-a77bbd8f-5c07-45c1-b6f4-5dc2ad439971 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea704b4947e8477cb261f3a61c786155'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:37.275124 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-a77bbd8f-5c07-45c1-b6f4-5dc2ad439971 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea704b4947e8477cb261f3a61c786155'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:37.288334 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-a77bbd8f-5c07-45c1-b6f4-5dc2ad439971 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea704b4947e8477cb261f3a61c786155'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:37.322346 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1197/2391] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:48:36 2022] DELETE /identity/v3/users/ea704b4947e8477cb261f3a61c786155 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:37.662009 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd7cbe-7eb9-4b86-a667-a5161adb0d4a None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:37.662406 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd7cbe-7eb9-4b86-a667-a5161adb0d4a None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:37.662788 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd7cbe-7eb9-4b86-a667-a5161adb0d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:37.682648 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-6ffd7cbe-7eb9-4b86-a667-a5161adb0d4a None None] MFA Rules not processed for user `e548ea934bc14b7e921f7e161c5496cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:37.686155 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-6ffd7cbe-7eb9-4b86-a667-a5161adb0d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:37.758778 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1195/2392] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:48:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:37.777134 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77bbd8f-5c07-45c1-b6f4-5dc2ad439971 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:37.779149 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-a77bbd8f-5c07-45c1-b6f4-5dc2ad439971 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:37.910242 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77bbd8f-5c07-45c1-b6f4-5dc2ad439971 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:37.918986 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd15b057-58df-4da5-8998-ee07dccd6f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:37.919652 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15b057-58df-4da5-8998-ee07dccd6f73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:37.919941 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15b057-58df-4da5-8998-ee07dccd6f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:37.920220 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15b057-58df-4da5-8998-ee07dccd6f73 admin admin] PATH_INFO: `/v3/projects/8a4186243afe47b5be831c9584eeef6d` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:37.922216 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd15b057-58df-4da5-8998-ee07dccd6f73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a4186243afe47b5be831c9584eeef6d)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:37.925117 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd15b057-58df-4da5-8998-ee07dccd6f73 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:38.089296 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-fd15b057-58df-4da5-8998-ee07dccd6f73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a4186243afe47b5be831c9584eeef6d', 'request_id': 'req-fd15b057-58df-4da5-8998-ee07dccd6f73'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:38.112263 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-fd15b057-58df-4da5-8998-ee07dccd6f73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a4186243afe47b5be831c9584eeef6d', 'request_id': 'req-fd15b057-58df-4da5-8998-ee07dccd6f73'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:38.160610 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:38.182939 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-fd15b057-58df-4da5-8998-ee07dccd6f73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a4186243afe47b5be831c9584eeef6d', 'request_id': 'req-fd15b057-58df-4da5-8998-ee07dccd6f73'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:38.201450 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-fd15b057-58df-4da5-8998-ee07dccd6f73 admin admin] The token cache is being invalidate because project 8a4186243afe47b5be831c9584eeef6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:48:38.201780 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-fd15b057-58df-4da5-8998-ee07dccd6f73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:38.202411 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-fd15b057-58df-4da5-8998-ee07dccd6f73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:38.210775 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1198/2393] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:48:37 2022] DELETE /identity/v3/projects/8a4186243afe47b5be831c9584eeef6d => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:38.213717 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffd7cbe-7eb9-4b86-a667-a5161adb0d4a None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:38.234655 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffd7cbe-7eb9-4b86-a667-a5161adb0d4a None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:38.339740 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0c0f67-2183-427a-b4db-690772a0a050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:38.341541 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c0f67-2183-427a-b4db-690772a0a050 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:38.341838 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c0f67-2183-427a-b4db-690772a0a050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:38.342119 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c0f67-2183-427a-b4db-690772a0a050 admin admin] PATH_INFO: `/v3/users/3e4ad7fc739c4dd6bec17d16969903e4` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:38.344355 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0c0f67-2183-427a-b4db-690772a0a050 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e4ad7fc739c4dd6bec17d16969903e4)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:38.347920 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0c0f67-2183-427a-b4db-690772a0a050 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:38.600389 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3b0c0f67-2183-427a-b4db-690772a0a050 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e4ad7fc739c4dd6bec17d16969903e4'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:38.720067 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3b0c0f67-2183-427a-b4db-690772a0a050 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e4ad7fc739c4dd6bec17d16969903e4'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:38.732952 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3b0c0f67-2183-427a-b4db-690772a0a050 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e4ad7fc739c4dd6bec17d16969903e4'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:38.764083 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3b0c0f67-2183-427a-b4db-690772a0a050 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e4ad7fc739c4dd6bec17d16969903e4'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:38.764083 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3b0c0f67-2183-427a-b4db-690772a0a050 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e4ad7fc739c4dd6bec17d16969903e4'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:38.772245 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1196/2394] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:48:38 2022] DELETE /identity/v3/users/3e4ad7fc739c4dd6bec17d16969903e4 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:39.146218 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd15b057-58df-4da5-8998-ee07dccd6f73 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:39.148439 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-fd15b057-58df-4da5-8998-ee07dccd6f73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:39.163685 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd15b057-58df-4da5-8998-ee07dccd6f73 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:39.239686 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe8aaeb-ea59-47f6-bdac-b37dec5185c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:39.240147 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe8aaeb-ea59-47f6-bdac-b37dec5185c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:39.240647 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe8aaeb-ea59-47f6-bdac-b37dec5185c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:39.241431 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe8aaeb-ea59-47f6-bdac-b37dec5185c0 admin admin] PATH_INFO: `/v3/projects/f266ab3819b346918360e68e11ae1d8f` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:39.243231 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe8aaeb-ea59-47f6-bdac-b37dec5185c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f266ab3819b346918360e68e11ae1d8f)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:39.246645 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe8aaeb-ea59-47f6-bdac-b37dec5185c0 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:39.311141 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0c0f67-2183-427a-b4db-690772a0a050 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:39.317588 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-3b0c0f67-2183-427a-b4db-690772a0a050 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:39.375591 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0c0f67-2183-427a-b4db-690772a0a050 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:39.379372 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f800481-04a7-4185-b988-4309767a6aca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:39.380055 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f800481-04a7-4185-b988-4309767a6aca service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:39.382853 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f800481-04a7-4185-b988-4309767a6aca service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:39.382853 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f800481-04a7-4185-b988-4309767a6aca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:39.382853 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-2f800481-04a7-4185-b988-4309767a6aca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:39.451486 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f800481-04a7-4185-b988-4309767a6aca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:39.453476 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f800481-04a7-4185-b988-4309767a6aca service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:39.462455 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-afe8aaeb-ea59-47f6-bdac-b37dec5185c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f266ab3819b346918360e68e11ae1d8f', 'request_id': 'req-afe8aaeb-ea59-47f6-bdac-b37dec5185c0'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:39.467274 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1197/2395] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:48:39 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:48:39.471712 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-afe8aaeb-ea59-47f6-bdac-b37dec5185c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f266ab3819b346918360e68e11ae1d8f', 'request_id': 'req-afe8aaeb-ea59-47f6-bdac-b37dec5185c0'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:39.608097 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-afe8aaeb-ea59-47f6-bdac-b37dec5185c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f266ab3819b346918360e68e11ae1d8f', 'request_id': 'req-afe8aaeb-ea59-47f6-bdac-b37dec5185c0'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:39.619825 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-afe8aaeb-ea59-47f6-bdac-b37dec5185c0 admin admin] The token cache is being invalidate because project f266ab3819b346918360e68e11ae1d8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:48:39.619825 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-afe8aaeb-ea59-47f6-bdac-b37dec5185c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:39.621060 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-afe8aaeb-ea59-47f6-bdac-b37dec5185c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:39.623095 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1199/2396] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:48:39 2022] DELETE /identity/v3/projects/f266ab3819b346918360e68e11ae1d8f => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:39.628900 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d3a19-7cbb-43f0-a01c-ae837498abc5 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:39.629563 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d3a19-7cbb-43f0-a01c-ae837498abc5 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:39.630373 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d3a19-7cbb-43f0-a01c-ae837498abc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:39.663932 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-199d3a19-7cbb-43f0-a01c-ae837498abc5 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:39.666423 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-199d3a19-7cbb-43f0-a01c-ae837498abc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:39.800584 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1198/2397] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:48:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:39.809507 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d84fa5-4fb2-482d-b53c-24a32d323dfb None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:39.809813 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d84fa5-4fb2-482d-b53c-24a32d323dfb None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:39.813454 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d84fa5-4fb2-482d-b53c-24a32d323dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:39.831837 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-44d84fa5-4fb2-482d-b53c-24a32d323dfb None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:39.833371 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-44d84fa5-4fb2-482d-b53c-24a32d323dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:39.848704 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1200/2398] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:48:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:39.853700 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d3a19-7cbb-43f0-a01c-ae837498abc5 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:39.863711 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d3a19-7cbb-43f0-a01c-ae837498abc5 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:39.877275 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c227f10-a2ec-4b55-9ee3-3f7b25906f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:39.877275 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c227f10-a2ec-4b55-9ee3-3f7b25906f96 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:39.877275 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c227f10-a2ec-4b55-9ee3-3f7b25906f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:39.883700 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c227f10-a2ec-4b55-9ee3-3f7b25906f96 admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:39.884925 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c227f10-a2ec-4b55-9ee3-3f7b25906f96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:48:39.885249 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c227f10-a2ec-4b55-9ee3-3f7b25906f96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:39.888936 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c227f10-a2ec-4b55-9ee3-3f7b25906f96 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:39.901469 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1199/2399] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 16:48:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:48:39.911253 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d84fa5-4fb2-482d-b53c-24a32d323dfb None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:39.919632 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d84fa5-4fb2-482d-b53c-24a32d323dfb None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:39.925792 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f6a0b-e122-4566-9438-0a346d6cf389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:39.926604 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f6a0b-e122-4566-9438-0a346d6cf389 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:39.927058 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f6a0b-e122-4566-9438-0a346d6cf389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:39.927356 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f6a0b-e122-4566-9438-0a346d6cf389 admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:39.928104 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643f6a0b-e122-4566-9438-0a346d6cf389 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:39.930074 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643f6a0b-e122-4566-9438-0a346d6cf389 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:40.080610 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1201/2400] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:48:39 2022] POST /identity/v3/projects => generated 389 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:48:40.084089 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c227f10-a2ec-4b55-9ee3-3f7b25906f96 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:40.094980 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c227f10-a2ec-4b55-9ee3-3f7b25906f96 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:40.159241 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd8ecdc-8a8e-4d56-8a9d-1a259b43bfa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:40.160185 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd8ecdc-8a8e-4d56-8a9d-1a259b43bfa3 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:40.160537 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd8ecdc-8a8e-4d56-8a9d-1a259b43bfa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:40.160883 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd8ecdc-8a8e-4d56-8a9d-1a259b43bfa3 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:40.161650 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd8ecdc-8a8e-4d56-8a9d-1a259b43bfa3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:40.163722 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd8ecdc-8a8e-4d56-8a9d-1a259b43bfa3 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:40.295213 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1200/2401] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:48:40 2022] POST /identity/v3/users => generated 295 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:48:40.300199 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f6a0b-e122-4566-9438-0a346d6cf389 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:40.317462 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f6a0b-e122-4566-9438-0a346d6cf389 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:40.325849 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe42186e-a6e4-4262-b9db-bfa3610bce7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:40.326643 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe42186e-a6e4-4262-b9db-bfa3610bce7d admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:40.326924 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe42186e-a6e4-4262-b9db-bfa3610bce7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:40.327197 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe42186e-a6e4-4262-b9db-bfa3610bce7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:40.327976 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe42186e-a6e4-4262-b9db-bfa3610bce7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:40.329979 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe42186e-a6e4-4262-b9db-bfa3610bce7d admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:40.342264 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1202/2402] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:48:40 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:48:40.346381 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd8ecdc-8a8e-4d56-8a9d-1a259b43bfa3 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:40.357388 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd8ecdc-8a8e-4d56-8a9d-1a259b43bfa3 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:40.362343 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cca49f1-ea8a-4d79-bf2f-6b3b862c159a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:40.363021 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca49f1-ea8a-4d79-bf2f-6b3b862c159a admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:40.363310 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca49f1-ea8a-4d79-bf2f-6b3b862c159a admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:40.363619 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca49f1-ea8a-4d79-bf2f-6b3b862c159a admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:40.364351 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cca49f1-ea8a-4d79-bf2f-6b3b862c159a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:40.365944 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cca49f1-ea8a-4d79-bf2f-6b3b862c159a admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:40.376999 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1201/2403] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:48:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:48:40.382471 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe42186e-a6e4-4262-b9db-bfa3610bce7d admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:40.398012 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe42186e-a6e4-4262-b9db-bfa3610bce7d admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:40.404059 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350ad965-9796-49ac-9a0a-eda88dfdbf69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:40.405117 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ad965-9796-49ac-9a0a-eda88dfdbf69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:40.405472 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ad965-9796-49ac-9a0a-eda88dfdbf69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:40.405820 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ad965-9796-49ac-9a0a-eda88dfdbf69 admin admin] PATH_INFO: `/v3/projects/1e8ab187215e4413abdd1475bd615692/users/d21fff8ab6f04299ab2cc50a36d9fd71/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:40.426280 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350ad965-9796-49ac-9a0a-eda88dfdbf69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e8ab187215e4413abdd1475bd615692, user_id=d21fff8ab6f04299ab2cc50a36d9fd71, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:40.429061 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350ad965-9796-49ac-9a0a-eda88dfdbf69 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:40.501736 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1203/2404] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:48:40 2022] PUT /identity/v3/projects/1e8ab187215e4413abdd1475bd615692/users/d21fff8ab6f04299ab2cc50a36d9fd71/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:40.512980 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d0434a-7e42-40a6-9b18-8a0ed170b9d8 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:40.513271 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d0434a-7e42-40a6-9b18-8a0ed170b9d8 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:40.513553 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d0434a-7e42-40a6-9b18-8a0ed170b9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:40.541359 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-d5d0434a-7e42-40a6-9b18-8a0ed170b9d8 None None] MFA Rules not processed for user `d21fff8ab6f04299ab2cc50a36d9fd71`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:40.542821 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d5d0434a-7e42-40a6-9b18-8a0ed170b9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:40.574284 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:40.611152 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1202/2405] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:48:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:40.622463 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350ad965-9796-49ac-9a0a-eda88dfdbf69 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:40.624273 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-350ad965-9796-49ac-9a0a-eda88dfdbf69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:40.659907 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:40.686121 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350ad965-9796-49ac-9a0a-eda88dfdbf69 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:40.689765 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc6721a-60cd-431b-bed7-e9580882b3c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:40.690519 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc6721a-60cd-431b-bed7-e9580882b3c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:40.690802 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc6721a-60cd-431b-bed7-e9580882b3c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:40.691081 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc6721a-60cd-431b-bed7-e9580882b3c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:40.700332 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc6721a-60cd-431b-bed7-e9580882b3c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:40.701764 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc6721a-60cd-431b-bed7-e9580882b3c8 service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:40.710895 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1204/2406] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:48:40 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:48:41.871501 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:42.638427 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:42.780566 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:44.260644 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:45.048787 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:52.094085 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d0434a-7e42-40a6-9b18-8a0ed170b9d8 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:52.095971 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d5d0434a-7e42-40a6-9b18-8a0ed170b9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:52.129994 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:52.170518 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d0434a-7e42-40a6-9b18-8a0ed170b9d8 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:52.177298 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdfd2b4-02be-4363-a609-67a14a46490f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:52.177298 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdfd2b4-02be-4363-a609-67a14a46490f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:52.177298 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdfd2b4-02be-4363-a609-67a14a46490f admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:52.177298 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdfd2b4-02be-4363-a609-67a14a46490f admin admin] PATH_INFO: `/v3/users/2c97711bd4244faa8a37bd669af2c369` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:52.178222 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdfd2b4-02be-4363-a609-67a14a46490f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c97711bd4244faa8a37bd669af2c369)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:52.180345 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdfd2b4-02be-4363-a609-67a14a46490f admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:52.412238 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-dbdfd2b4-02be-4363-a609-67a14a46490f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c97711bd4244faa8a37bd669af2c369'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:52.486566 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-dbdfd2b4-02be-4363-a609-67a14a46490f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c97711bd4244faa8a37bd669af2c369'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:52.494134 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-dbdfd2b4-02be-4363-a609-67a14a46490f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c97711bd4244faa8a37bd669af2c369'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:52.506039 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-dbdfd2b4-02be-4363-a609-67a14a46490f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c97711bd4244faa8a37bd669af2c369'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:52.506694 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-dbdfd2b4-02be-4363-a609-67a14a46490f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c97711bd4244faa8a37bd669af2c369'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:52.512538 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1203/2407] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:48:52 2022] DELETE /identity/v3/users/2c97711bd4244faa8a37bd669af2c369 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:52.517668 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192cc3e8-a1e1-4311-85f1-46370a6d8a46 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:52.518175 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192cc3e8-a1e1-4311-85f1-46370a6d8a46 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:52.518272 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192cc3e8-a1e1-4311-85f1-46370a6d8a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:52.537581 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-192cc3e8-a1e1-4311-85f1-46370a6d8a46 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:52.538840 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-192cc3e8-a1e1-4311-85f1-46370a6d8a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:52.598265 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1205/2408] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:48:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:52.608153 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdfd2b4-02be-4363-a609-67a14a46490f admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:52.609624 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-dbdfd2b4-02be-4363-a609-67a14a46490f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:52.667111 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdfd2b4-02be-4363-a609-67a14a46490f admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:52.670838 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c9ff3e-acef-459f-92d7-c88b74105ca5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:52.671486 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c9ff3e-acef-459f-92d7-c88b74105ca5 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:52.671818 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c9ff3e-acef-459f-92d7-c88b74105ca5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:52.672087 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c9ff3e-acef-459f-92d7-c88b74105ca5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:52.681961 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c9ff3e-acef-459f-92d7-c88b74105ca5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:52.683409 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c9ff3e-acef-459f-92d7-c88b74105ca5 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:52.693437 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1204/2409] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:48:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:48:53.216163 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192cc3e8-a1e1-4311-85f1-46370a6d8a46 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:53.217780 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-192cc3e8-a1e1-4311-85f1-46370a6d8a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:53.230121 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192cc3e8-a1e1-4311-85f1-46370a6d8a46 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:53.237388 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bba746-b791-4d63-94c3-7886d737aa90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:53.238515 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bba746-b791-4d63-94c3-7886d737aa90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:53.239733 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bba746-b791-4d63-94c3-7886d737aa90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:53.239733 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bba746-b791-4d63-94c3-7886d737aa90 admin admin] PATH_INFO: `/v3/projects/45099f0404f5499b9aaa76c3e79a65b6` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:53.240412 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bba746-b791-4d63-94c3-7886d737aa90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45099f0404f5499b9aaa76c3e79a65b6)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:53.242351 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bba746-b791-4d63-94c3-7886d737aa90 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:53.420002 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-81bba746-b791-4d63-94c3-7886d737aa90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45099f0404f5499b9aaa76c3e79a65b6', 'request_id': 'req-81bba746-b791-4d63-94c3-7886d737aa90'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:53.427222 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-81bba746-b791-4d63-94c3-7886d737aa90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45099f0404f5499b9aaa76c3e79a65b6', 'request_id': 'req-81bba746-b791-4d63-94c3-7886d737aa90'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:53.505796 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-81bba746-b791-4d63-94c3-7886d737aa90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45099f0404f5499b9aaa76c3e79a65b6', 'request_id': 'req-81bba746-b791-4d63-94c3-7886d737aa90'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:53.517137 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-81bba746-b791-4d63-94c3-7886d737aa90 admin admin] The token cache is being invalidate because project 45099f0404f5499b9aaa76c3e79a65b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:48:53.517426 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-81bba746-b791-4d63-94c3-7886d737aa90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:53.518082 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-81bba746-b791-4d63-94c3-7886d737aa90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:53.519726 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1206/2410] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:48:53 2022] DELETE /identity/v3/projects/45099f0404f5499b9aaa76c3e79a65b6 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:53.523775 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c9ff3e-acef-459f-92d7-c88b74105ca5 service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:53.535014 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c9ff3e-acef-459f-92d7-c88b74105ca5 service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:53.593885 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:53.606457 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675d074a-92e5-47e0-a864-3a9fc958f876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:53.607436 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d074a-92e5-47e0-a864-3a9fc958f876 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:53.607858 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d074a-92e5-47e0-a864-3a9fc958f876 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:53.608145 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d074a-92e5-47e0-a864-3a9fc958f876 admin admin] PATH_INFO: `/v3/users/3419c2d629e944b6b7cd738276181795` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:53.609344 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675d074a-92e5-47e0-a864-3a9fc958f876 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3419c2d629e944b6b7cd738276181795)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:53.611283 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675d074a-92e5-47e0-a864-3a9fc958f876 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:53.811579 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-675d074a-92e5-47e0-a864-3a9fc958f876 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3419c2d629e944b6b7cd738276181795'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:53.877398 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-675d074a-92e5-47e0-a864-3a9fc958f876 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3419c2d629e944b6b7cd738276181795'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:53.885205 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-675d074a-92e5-47e0-a864-3a9fc958f876 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3419c2d629e944b6b7cd738276181795'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:53.899220 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-675d074a-92e5-47e0-a864-3a9fc958f876 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3419c2d629e944b6b7cd738276181795'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:53.899972 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-675d074a-92e5-47e0-a864-3a9fc958f876 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3419c2d629e944b6b7cd738276181795'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:53.905964 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1205/2411] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:48:53 2022] DELETE /identity/v3/users/3419c2d629e944b6b7cd738276181795 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:54.345533 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bba746-b791-4d63-94c3-7886d737aa90 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:54.353796 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-81bba746-b791-4d63-94c3-7886d737aa90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:54.373173 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bba746-b791-4d63-94c3-7886d737aa90 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:54.386580 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:54.434441 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e3baa4-f644-4d85-abab-5da557613b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:54.436239 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3baa4-f644-4d85-abab-5da557613b2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:54.436532 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3baa4-f644-4d85-abab-5da557613b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:54.436811 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3baa4-f644-4d85-abab-5da557613b2b admin admin] PATH_INFO: `/v3/projects/cbc2045be1b84df6889fd32e0f134eb7` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:54.438102 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e3baa4-f644-4d85-abab-5da557613b2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbc2045be1b84df6889fd32e0f134eb7)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:54.440163 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e3baa4-f644-4d85-abab-5da557613b2b admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:54.541467 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-d2e3baa4-f644-4d85-abab-5da557613b2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbc2045be1b84df6889fd32e0f134eb7', 'request_id': 'req-d2e3baa4-f644-4d85-abab-5da557613b2b'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:54.548384 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-d2e3baa4-f644-4d85-abab-5da557613b2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbc2045be1b84df6889fd32e0f134eb7', 'request_id': 'req-d2e3baa4-f644-4d85-abab-5da557613b2b'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:54.643328 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-d2e3baa4-f644-4d85-abab-5da557613b2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbc2045be1b84df6889fd32e0f134eb7', 'request_id': 'req-d2e3baa4-f644-4d85-abab-5da557613b2b'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:54.655478 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-d2e3baa4-f644-4d85-abab-5da557613b2b admin admin] The token cache is being invalidate because project cbc2045be1b84df6889fd32e0f134eb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:48:54.658169 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-d2e3baa4-f644-4d85-abab-5da557613b2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:54.658169 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-d2e3baa4-f644-4d85-abab-5da557613b2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:54.658541 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1207/2412] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:48:54 2022] DELETE /identity/v3/projects/cbc2045be1b84df6889fd32e0f134eb7 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:54.664527 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ade89-d9a4-447b-a8fe-72ff8c743db7 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:54.665003 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ade89-d9a4-447b-a8fe-72ff8c743db7 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:54.665111 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ade89-d9a4-447b-a8fe-72ff8c743db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:54.684342 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-562ade89-d9a4-447b-a8fe-72ff8c743db7 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:54.686852 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-562ade89-d9a4-447b-a8fe-72ff8c743db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:54.745844 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1206/2413] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:48:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:54.754514 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e7db1-4452-45db-9f4d-888c436e3dbd None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:54.754802 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e7db1-4452-45db-9f4d-888c436e3dbd None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:54.755081 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e7db1-4452-45db-9f4d-888c436e3dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:54.774149 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-385e7db1-4452-45db-9f4d-888c436e3dbd None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:54.775358 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-385e7db1-4452-45db-9f4d-888c436e3dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:54.784765 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1208/2414] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:48:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:54.789348 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ade89-d9a4-447b-a8fe-72ff8c743db7 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:54.800216 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ade89-d9a4-447b-a8fe-72ff8c743db7 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:54.805033 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90793739-b0fa-4c49-81ff-498d5a88d716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:54.805718 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90793739-b0fa-4c49-81ff-498d5a88d716 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:54.805997 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90793739-b0fa-4c49-81ff-498d5a88d716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:54.806273 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90793739-b0fa-4c49-81ff-498d5a88d716 admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:54.806999 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90793739-b0fa-4c49-81ff-498d5a88d716 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:48:54.807303 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90793739-b0fa-4c49-81ff-498d5a88d716 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:54.808917 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90793739-b0fa-4c49-81ff-498d5a88d716 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:54.822864 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1207/2415] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 16:48:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:48:54.828190 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385e7db1-4452-45db-9f4d-888c436e3dbd None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:54.837910 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385e7db1-4452-45db-9f4d-888c436e3dbd None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:54.843161 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0be2954-5c52-4689-8259-9dab14a901fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:54.843858 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be2954-5c52-4689-8259-9dab14a901fe admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:54.844140 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be2954-5c52-4689-8259-9dab14a901fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:54.844412 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be2954-5c52-4689-8259-9dab14a901fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:54.845101 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0be2954-5c52-4689-8259-9dab14a901fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:54.846993 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0be2954-5c52-4689-8259-9dab14a901fe admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:54.953506 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1209/2416] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:48:54 2022] POST /identity/v3/projects => generated 375 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:48:54.957798 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90793739-b0fa-4c49-81ff-498d5a88d716 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:54.968494 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90793739-b0fa-4c49-81ff-498d5a88d716 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:55.016983 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:55.026450 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3827b6d-43d6-4aed-85e2-def5f3e61c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:55.031635 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3827b6d-43d6-4aed-85e2-def5f3e61c5e admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:55.031635 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3827b6d-43d6-4aed-85e2-def5f3e61c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:55.031635 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3827b6d-43d6-4aed-85e2-def5f3e61c5e admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:55.031635 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3827b6d-43d6-4aed-85e2-def5f3e61c5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:55.031635 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3827b6d-43d6-4aed-85e2-def5f3e61c5e admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:55.135638 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1208/2417] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:48:54 2022] POST /identity/v3/users => generated 288 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:48:55.139941 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0be2954-5c52-4689-8259-9dab14a901fe admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:55.150328 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0be2954-5c52-4689-8259-9dab14a901fe admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:55.155596 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0661fa76-b68a-460f-b1cf-b137702f1581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:55.156273 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0661fa76-b68a-460f-b1cf-b137702f1581 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:55.156554 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0661fa76-b68a-460f-b1cf-b137702f1581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:55.156826 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0661fa76-b68a-460f-b1cf-b137702f1581 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:55.157577 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0661fa76-b68a-460f-b1cf-b137702f1581 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:55.159160 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0661fa76-b68a-460f-b1cf-b137702f1581 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:55.171057 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1210/2418] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:48:55.175137 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3827b6d-43d6-4aed-85e2-def5f3e61c5e admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:55.185188 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3827b6d-43d6-4aed-85e2-def5f3e61c5e admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:55.194409 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54888161-4d93-4b93-abed-077ba4ea32bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:55.195075 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54888161-4d93-4b93-abed-077ba4ea32bf admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:55.195361 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54888161-4d93-4b93-abed-077ba4ea32bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:55.195675 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54888161-4d93-4b93-abed-077ba4ea32bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:55.196406 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54888161-4d93-4b93-abed-077ba4ea32bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:55.197997 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54888161-4d93-4b93-abed-077ba4ea32bf admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:55.210022 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1209/2419] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:48:55.213746 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0661fa76-b68a-460f-b1cf-b137702f1581 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:55.224910 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0661fa76-b68a-460f-b1cf-b137702f1581 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:55.229626 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c18f393-8ce0-4f6b-882e-0ac5523e2a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:55.230507 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18f393-8ce0-4f6b-882e-0ac5523e2a36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:55.230794 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18f393-8ce0-4f6b-882e-0ac5523e2a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:55.231105 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18f393-8ce0-4f6b-882e-0ac5523e2a36 admin admin] PATH_INFO: `/v3/projects/f087575b3b0443398b25985b720a461b/users/8be47d3d70904815bc6e22c5e9b0b5af/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:55.246606 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c18f393-8ce0-4f6b-882e-0ac5523e2a36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f087575b3b0443398b25985b720a461b, user_id=8be47d3d70904815bc6e22c5e9b0b5af, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:55.249648 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c18f393-8ce0-4f6b-882e-0ac5523e2a36 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:55.350005 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1211/2420] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:48:55 2022] PUT /identity/v3/projects/f087575b3b0443398b25985b720a461b/users/8be47d3d70904815bc6e22c5e9b0b5af/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:55.363019 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e8b298-e20d-4cf8-ae0c-6fcaad12cc16 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:55.363313 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e8b298-e20d-4cf8-ae0c-6fcaad12cc16 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:55.363634 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e8b298-e20d-4cf8-ae0c-6fcaad12cc16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:55.392885 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-01e8b298-e20d-4cf8-ae0c-6fcaad12cc16 None None] MFA Rules not processed for user `8be47d3d70904815bc6e22c5e9b0b5af`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:55.394098 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-01e8b298-e20d-4cf8-ae0c-6fcaad12cc16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:55.458843 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1210/2421] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:48:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:55.463763 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c18f393-8ce0-4f6b-882e-0ac5523e2a36 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:55.512265 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:55.534226 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c18f393-8ce0-4f6b-882e-0ac5523e2a36 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:55.539044 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8314cada-a846-4659-8fb7-e961b3625d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:55.539766 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8314cada-a846-4659-8fb7-e961b3625d58 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:55.540025 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8314cada-a846-4659-8fb7-e961b3625d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:55.540300 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8314cada-a846-4659-8fb7-e961b3625d58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:55.541142 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8314cada-a846-4659-8fb7-e961b3625d58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:55.542947 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8314cada-a846-4659-8fb7-e961b3625d58 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:55.649221 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1212/2422] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:48:55 2022] POST /identity/v3/projects => generated 375 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:48:55.650504 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e8b298-e20d-4cf8-ae0c-6fcaad12cc16 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:55.662647 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e8b298-e20d-4cf8-ae0c-6fcaad12cc16 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:55.724688 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ad1e0e-2547-44a4-a94e-1f8fbad9511b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:55.725535 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad1e0e-2547-44a4-a94e-1f8fbad9511b admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:55.725535 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad1e0e-2547-44a4-a94e-1f8fbad9511b admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:55.725720 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad1e0e-2547-44a4-a94e-1f8fbad9511b admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:55.726406 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ad1e0e-2547-44a4-a94e-1f8fbad9511b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:55.729592 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ad1e0e-2547-44a4-a94e-1f8fbad9511b admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:55.809554 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1211/2423] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:48:55 2022] POST /identity/v3/users => generated 294 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:48:55.814334 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8314cada-a846-4659-8fb7-e961b3625d58 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:55.825721 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8314cada-a846-4659-8fb7-e961b3625d58 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:55.836965 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d153e7-c589-436d-8d8c-6982682ddcaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:55.837797 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d153e7-c589-436d-8d8c-6982682ddcaf admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:55.838181 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d153e7-c589-436d-8d8c-6982682ddcaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:55.838558 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d153e7-c589-436d-8d8c-6982682ddcaf admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:55.839394 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d153e7-c589-436d-8d8c-6982682ddcaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:55.841146 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d153e7-c589-436d-8d8c-6982682ddcaf admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:55.856371 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1213/2424] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:48:55.861356 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ad1e0e-2547-44a4-a94e-1f8fbad9511b admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:55.873342 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ad1e0e-2547-44a4-a94e-1f8fbad9511b admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:55.878282 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a71ab7-2d1b-43f7-92f2-d6669ad9f79e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:55.879227 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a71ab7-2d1b-43f7-92f2-d6669ad9f79e admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:55.879532 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a71ab7-2d1b-43f7-92f2-d6669ad9f79e admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:55.879811 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a71ab7-2d1b-43f7-92f2-d6669ad9f79e admin admin] PATH_INFO: `/v3/projects/8f8e05ade18c4994ac977be07eb706f5/users/9ee96a6b0c4b4b2a8ca03cf90ee617b1/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:55.897697 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a71ab7-2d1b-43f7-92f2-d6669ad9f79e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f8e05ade18c4994ac977be07eb706f5, user_id=9ee96a6b0c4b4b2a8ca03cf90ee617b1, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:55.900386 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a71ab7-2d1b-43f7-92f2-d6669ad9f79e admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:55.998209 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1212/2425] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:48:55 2022] PUT /identity/v3/projects/8f8e05ade18c4994ac977be07eb706f5/users/9ee96a6b0c4b4b2a8ca03cf90ee617b1/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:55.998490 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d153e7-c589-436d-8d8c-6982682ddcaf admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:56.006734 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d153e7-c589-436d-8d8c-6982682ddcaf admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:56.069927 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c529024-d9fd-407b-ae3e-25f6c684dca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:56.070776 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c529024-d9fd-407b-ae3e-25f6c684dca9 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:56.071129 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c529024-d9fd-407b-ae3e-25f6c684dca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:56.071367 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c529024-d9fd-407b-ae3e-25f6c684dca9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:56.072216 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c529024-d9fd-407b-ae3e-25f6c684dca9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:56.073793 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c529024-d9fd-407b-ae3e-25f6c684dca9 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:56.086476 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1214/2426] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:48:56.090980 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a71ab7-2d1b-43f7-92f2-d6669ad9f79e admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:56.102836 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a71ab7-2d1b-43f7-92f2-d6669ad9f79e admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:56.107925 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b12d50-289f-4ee6-9b4d-d4bf08ed33ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:56.108855 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b12d50-289f-4ee6-9b4d-d4bf08ed33ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:56.109142 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b12d50-289f-4ee6-9b4d-d4bf08ed33ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:56.109416 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b12d50-289f-4ee6-9b4d-d4bf08ed33ea admin admin] PATH_INFO: `/v3/projects/8f8e05ade18c4994ac977be07eb706f5/users/9ee96a6b0c4b4b2a8ca03cf90ee617b1/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:56.111457 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b12d50-289f-4ee6-9b4d-d4bf08ed33ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f8e05ade18c4994ac977be07eb706f5, user_id=9ee96a6b0c4b4b2a8ca03cf90ee617b1, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:56.114230 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b12d50-289f-4ee6-9b4d-d4bf08ed33ea admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:56.126665 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1213/2427] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:48:56 2022] PUT /identity/v3/projects/8f8e05ade18c4994ac977be07eb706f5/users/9ee96a6b0c4b4b2a8ca03cf90ee617b1/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:56.147552 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072cc666-40b9-4743-855f-427114b9b2bf None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:56.147906 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072cc666-40b9-4743-855f-427114b9b2bf None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:56.148236 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072cc666-40b9-4743-855f-427114b9b2bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:56.178959 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-072cc666-40b9-4743-855f-427114b9b2bf None None] MFA Rules not processed for user `9ee96a6b0c4b4b2a8ca03cf90ee617b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:56.180232 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-072cc666-40b9-4743-855f-427114b9b2bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:56.255809 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1215/2428] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:48:56 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:56.268167 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b12d50-289f-4ee6-9b4d-d4bf08ed33ea admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:56.270084 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-21b12d50-289f-4ee6-9b4d-d4bf08ed33ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:56.286617 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:56.332608 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b12d50-289f-4ee6-9b4d-d4bf08ed33ea admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:56.336299 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8185a-2675-441b-aba0-769fac9a2021 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:56.338859 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8185a-2675-441b-aba0-769fac9a2021 service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:56.338859 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8185a-2675-441b-aba0-769fac9a2021 service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:56.338859 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8185a-2675-441b-aba0-769fac9a2021 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:56.400804 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f8185a-2675-441b-aba0-769fac9a2021 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:56.402380 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f8185a-2675-441b-aba0-769fac9a2021 service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:56.412004 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1214/2429] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:48:56 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:48:56.867280 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072cc666-40b9-4743-855f-427114b9b2bf None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:56.869860 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-072cc666-40b9-4743-855f-427114b9b2bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:56.948507 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:57.000767 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072cc666-40b9-4743-855f-427114b9b2bf None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:57.013598 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd04dacb-78d6-4891-857a-e4fb0808a0c8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70f704a231a5474e8067a6c2dd88b355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:57.014430 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd04dacb-78d6-4891-857a-e4fb0808a0c8 service placement] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:57.014735 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd04dacb-78d6-4891-857a-e4fb0808a0c8 service placement] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:57.014994 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd04dacb-78d6-4891-857a-e4fb0808a0c8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:57.017297 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-cd04dacb-78d6-4891-857a-e4fb0808a0c8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:57.032150 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd04dacb-78d6-4891-857a-e4fb0808a0c8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:57.035621 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd04dacb-78d6-4891-857a-e4fb0808a0c8 service placement] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:57.052639 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1216/2430] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:48:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:48:59.573790 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8185a-2675-441b-aba0-769fac9a2021 service cinder] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:59.638452 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8185a-2675-441b-aba0-769fac9a2021 service cinder] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:59.642956 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab9cf55-92e4-4ca2-b408-1d1429683a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:59.643965 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab9cf55-92e4-4ca2-b408-1d1429683a99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:59.644256 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab9cf55-92e4-4ca2-b408-1d1429683a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:59.644536 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab9cf55-92e4-4ca2-b408-1d1429683a99 admin admin] PATH_INFO: `/v3/users/8be47d3d70904815bc6e22c5e9b0b5af` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:59.645751 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab9cf55-92e4-4ca2-b408-1d1429683a99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8be47d3d70904815bc6e22c5e9b0b5af)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:59.647708 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab9cf55-92e4-4ca2-b408-1d1429683a99 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:59.851015 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3ab9cf55-92e4-4ca2-b408-1d1429683a99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8be47d3d70904815bc6e22c5e9b0b5af'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:59.937941 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3ab9cf55-92e4-4ca2-b408-1d1429683a99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8be47d3d70904815bc6e22c5e9b0b5af'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:59.945609 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3ab9cf55-92e4-4ca2-b408-1d1429683a99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8be47d3d70904815bc6e22c5e9b0b5af'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:59.951763 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:59.958994 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3ab9cf55-92e4-4ca2-b408-1d1429683a99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8be47d3d70904815bc6e22c5e9b0b5af'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:59.959711 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-3ab9cf55-92e4-4ca2-b408-1d1429683a99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8be47d3d70904815bc6e22c5e9b0b5af'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:48:59.965628 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1215/2431] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:48:59 2022] DELETE /identity/v3/users/8be47d3d70904815bc6e22c5e9b0b5af => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:48:59.970818 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e7a41-474a-46ca-af57-1fa814c73e4a None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:59.971289 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e7a41-474a-46ca-af57-1fa814c73e4a None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:59.971381 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e7a41-474a-46ca-af57-1fa814c73e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:59.989140 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-f57e7a41-474a-46ca-af57-1fa814c73e4a None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:59.990343 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-f57e7a41-474a-46ca-af57-1fa814c73e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:00.050482 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1217/2432] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:48:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:49:00.060308 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab9cf55-92e4-4ca2-b408-1d1429683a99 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:00.061926 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-3ab9cf55-92e4-4ca2-b408-1d1429683a99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:00.132049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab9cf55-92e4-4ca2-b408-1d1429683a99 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:00.135792 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1bc5a1-3985-423f-a6ee-05203927dd6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:00.136465 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1bc5a1-3985-423f-a6ee-05203927dd6f service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:00.136758 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1bc5a1-3985-423f-a6ee-05203927dd6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:00.137032 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1bc5a1-3985-423f-a6ee-05203927dd6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:00.149518 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1bc5a1-3985-423f-a6ee-05203927dd6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:00.149518 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1bc5a1-3985-423f-a6ee-05203927dd6f service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:00.159516 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1216/2433] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:49:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:49:00.871709 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57e7a41-474a-46ca-af57-1fa814c73e4a None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:00.874127 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-f57e7a41-474a-46ca-af57-1fa814c73e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:00.886390 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57e7a41-474a-46ca-af57-1fa814c73e4a None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:00.904786 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d8c8a5-dd2a-4284-9d76-9ac00e1abf85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:00.905855 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8c8a5-dd2a-4284-9d76-9ac00e1abf85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:00.906139 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8c8a5-dd2a-4284-9d76-9ac00e1abf85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:00.906412 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8c8a5-dd2a-4284-9d76-9ac00e1abf85 admin admin] PATH_INFO: `/v3/projects/f087575b3b0443398b25985b720a461b` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:00.910070 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d8c8a5-dd2a-4284-9d76-9ac00e1abf85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f087575b3b0443398b25985b720a461b)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:00.912316 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d8c8a5-dd2a-4284-9d76-9ac00e1abf85 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:01.020339 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-53d8c8a5-dd2a-4284-9d76-9ac00e1abf85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f087575b3b0443398b25985b720a461b', 'request_id': 'req-53d8c8a5-dd2a-4284-9d76-9ac00e1abf85'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:01.027485 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-53d8c8a5-dd2a-4284-9d76-9ac00e1abf85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f087575b3b0443398b25985b720a461b', 'request_id': 'req-53d8c8a5-dd2a-4284-9d76-9ac00e1abf85'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:01.103944 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-53d8c8a5-dd2a-4284-9d76-9ac00e1abf85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f087575b3b0443398b25985b720a461b', 'request_id': 'req-53d8c8a5-dd2a-4284-9d76-9ac00e1abf85'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:01.115187 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-53d8c8a5-dd2a-4284-9d76-9ac00e1abf85 admin admin] The token cache is being invalidate because project f087575b3b0443398b25985b720a461b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:49:01.115471 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-53d8c8a5-dd2a-4284-9d76-9ac00e1abf85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:01.116115 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-53d8c8a5-dd2a-4284-9d76-9ac00e1abf85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:01.117655 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1218/2434] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:49:00 2022] DELETE /identity/v3/projects/f087575b3b0443398b25985b720a461b => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:49:01.121697 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1bc5a1-3985-423f-a6ee-05203927dd6f service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:01.133125 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1bc5a1-3985-423f-a6ee-05203927dd6f service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:01.208901 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d22b2a6-d14d-45a8-8a30-8e266448f43b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:01.209962 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d22b2a6-d14d-45a8-8a30-8e266448f43b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:01.210243 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d22b2a6-d14d-45a8-8a30-8e266448f43b admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:01.210531 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d22b2a6-d14d-45a8-8a30-8e266448f43b admin admin] PATH_INFO: `/v3/users/9ee96a6b0c4b4b2a8ca03cf90ee617b1` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:01.212332 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d22b2a6-d14d-45a8-8a30-8e266448f43b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ee96a6b0c4b4b2a8ca03cf90ee617b1)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:01.218051 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d22b2a6-d14d-45a8-8a30-8e266448f43b admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:01.434642 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-5d22b2a6-d14d-45a8-8a30-8e266448f43b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ee96a6b0c4b4b2a8ca03cf90ee617b1'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:01.505934 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-5d22b2a6-d14d-45a8-8a30-8e266448f43b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ee96a6b0c4b4b2a8ca03cf90ee617b1'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:01.513851 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-5d22b2a6-d14d-45a8-8a30-8e266448f43b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ee96a6b0c4b4b2a8ca03cf90ee617b1'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:01.526807 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-5d22b2a6-d14d-45a8-8a30-8e266448f43b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ee96a6b0c4b4b2a8ca03cf90ee617b1'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:01.527837 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-5d22b2a6-d14d-45a8-8a30-8e266448f43b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ee96a6b0c4b4b2a8ca03cf90ee617b1'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:01.533934 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1217/2435] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:49:01 2022] DELETE /identity/v3/users/9ee96a6b0c4b4b2a8ca03cf90ee617b1 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:49:02.016362 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d8c8a5-dd2a-4284-9d76-9ac00e1abf85 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:02.020529 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-53d8c8a5-dd2a-4284-9d76-9ac00e1abf85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:02.034458 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d8c8a5-dd2a-4284-9d76-9ac00e1abf85 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:02.084639 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:49:02.095697 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc2a329-437d-489d-a26c-1bdb02235d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:02.096746 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2a329-437d-489d-a26c-1bdb02235d0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:02.097084 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2a329-437d-489d-a26c-1bdb02235d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:02.097364 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2a329-437d-489d-a26c-1bdb02235d0a admin admin] PATH_INFO: `/v3/projects/8f8e05ade18c4994ac977be07eb706f5` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:02.098566 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc2a329-437d-489d-a26c-1bdb02235d0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f8e05ade18c4994ac977be07eb706f5)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:02.100560 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc2a329-437d-489d-a26c-1bdb02235d0a admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:02.222152 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-1dc2a329-437d-489d-a26c-1bdb02235d0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f8e05ade18c4994ac977be07eb706f5', 'request_id': 'req-1dc2a329-437d-489d-a26c-1bdb02235d0a'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:02.228984 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-1dc2a329-437d-489d-a26c-1bdb02235d0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f8e05ade18c4994ac977be07eb706f5', 'request_id': 'req-1dc2a329-437d-489d-a26c-1bdb02235d0a'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:02.243840 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d22b2a6-d14d-45a8-8a30-8e266448f43b admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:02.245512 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-5d22b2a6-d14d-45a8-8a30-8e266448f43b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:02.259855 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d22b2a6-d14d-45a8-8a30-8e266448f43b admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:02.265260 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:49:02.318522 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8909fc6-5ae6-4b13-af29-a1ec8591dbb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:02.319464 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8909fc6-5ae6-4b13-af29-a1ec8591dbb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:02.319777 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8909fc6-5ae6-4b13-af29-a1ec8591dbb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:02.320058 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8909fc6-5ae6-4b13-af29-a1ec8591dbb5 admin admin] PATH_INFO: `/v3/users/d21fff8ab6f04299ab2cc50a36d9fd71` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:02.321263 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8909fc6-5ae6-4b13-af29-a1ec8591dbb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d21fff8ab6f04299ab2cc50a36d9fd71)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:02.323190 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8909fc6-5ae6-4b13-af29-a1ec8591dbb5 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:02.406620 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-1dc2a329-437d-489d-a26c-1bdb02235d0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f8e05ade18c4994ac977be07eb706f5', 'request_id': 'req-1dc2a329-437d-489d-a26c-1bdb02235d0a'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:02.418259 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-1dc2a329-437d-489d-a26c-1bdb02235d0a admin admin] The token cache is being invalidate because project 8f8e05ade18c4994ac977be07eb706f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:49:02.418592 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-1dc2a329-437d-489d-a26c-1bdb02235d0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:02.419418 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-1dc2a329-437d-489d-a26c-1bdb02235d0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:02.421369 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1219/2436] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:49:02 2022] DELETE /identity/v3/projects/8f8e05ade18c4994ac977be07eb706f5 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:49:02.426124 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807cf78-ad55-4b64-82ef-fbad5fcacbf0 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:02.426410 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807cf78-ad55-4b64-82ef-fbad5fcacbf0 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:02.426684 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807cf78-ad55-4b64-82ef-fbad5fcacbf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:02.444799 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-c807cf78-ad55-4b64-82ef-fbad5fcacbf0 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:49:02.455782 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-c807cf78-ad55-4b64-82ef-fbad5fcacbf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:02.456297 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1220/2437] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:49:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:49:02.465292 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c59610-b392-44c0-b015-cbca39928b9b None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:02.465572 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c59610-b392-44c0-b015-cbca39928b9b None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:02.465844 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c59610-b392-44c0-b015-cbca39928b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:02.485844 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-21c59610-b392-44c0-b015-cbca39928b9b None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:49:02.487084 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-21c59610-b392-44c0-b015-cbca39928b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:02.495445 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1221/2438] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:49:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:49:02.499893 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c59610-b392-44c0-b015-cbca39928b9b None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:02.509337 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c59610-b392-44c0-b015-cbca39928b9b None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:02.514627 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a882927-2746-4f14-9261-680bb9f59d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:02.515206 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a882927-2746-4f14-9261-680bb9f59d05 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:02.515526 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a882927-2746-4f14-9261-680bb9f59d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:02.515802 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a882927-2746-4f14-9261-680bb9f59d05 admin admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:02.516516 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a882927-2746-4f14-9261-680bb9f59d05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:49:02.516818 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a882927-2746-4f14-9261-680bb9f59d05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:02.518437 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a882927-2746-4f14-9261-680bb9f59d05 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:02.530802 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1222/2439] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 16:49:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:49:02.535081 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a882927-2746-4f14-9261-680bb9f59d05 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:02.544746 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a882927-2746-4f14-9261-680bb9f59d05 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:02.549831 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5d880f-a2e8-46df-9dbf-fa5be36a6b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:02.551114 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d880f-a2e8-46df-9dbf-fa5be36a6b36 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:02.551425 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d880f-a2e8-46df-9dbf-fa5be36a6b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:02.551734 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d880f-a2e8-46df-9dbf-fa5be36a6b36 admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:02.552421 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5d880f-a2e8-46df-9dbf-fa5be36a6b36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:02.554332 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5d880f-a2e8-46df-9dbf-fa5be36a6b36 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:02.609553 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b8909fc6-5ae6-4b13-af29-a1ec8591dbb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd21fff8ab6f04299ab2cc50a36d9fd71'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:02.686529 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1223/2440] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:49:02 2022] POST /identity/v3/projects => generated 377 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:49:02.690737 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5d880f-a2e8-46df-9dbf-fa5be36a6b36 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:02.715444 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b8909fc6-5ae6-4b13-af29-a1ec8591dbb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd21fff8ab6f04299ab2cc50a36d9fd71'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:02.725977 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b8909fc6-5ae6-4b13-af29-a1ec8591dbb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd21fff8ab6f04299ab2cc50a36d9fd71'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:02.739707 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b8909fc6-5ae6-4b13-af29-a1ec8591dbb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd21fff8ab6f04299ab2cc50a36d9fd71'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:02.740444 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-b8909fc6-5ae6-4b13-af29-a1ec8591dbb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd21fff8ab6f04299ab2cc50a36d9fd71'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:02.747095 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1218/2441] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:49:02 2022] DELETE /identity/v3/users/d21fff8ab6f04299ab2cc50a36d9fd71 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:49:02.752378 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7aebd-bc41-4513-a069-bb3a66ec65af None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:02.752814 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7aebd-bc41-4513-a069-bb3a66ec65af None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:02.753101 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7aebd-bc41-4513-a069-bb3a66ec65af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:02.766974 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5d880f-a2e8-46df-9dbf-fa5be36a6b36 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:02.772263 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-faf7aebd-bc41-4513-a069-bb3a66ec65af None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:49:02.773656 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-faf7aebd-bc41-4513-a069-bb3a66ec65af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:02.820018 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:49:02.851588 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62764b97-c69e-4487-848f-b61484488eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:02.852278 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62764b97-c69e-4487-848f-b61484488eaa admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:02.852566 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62764b97-c69e-4487-848f-b61484488eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:02.852846 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62764b97-c69e-4487-848f-b61484488eaa admin admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:02.853702 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62764b97-c69e-4487-848f-b61484488eaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:02.855782 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62764b97-c69e-4487-848f-b61484488eaa admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:02.856036 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1219/2442] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:49:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:49:02.865747 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf7aebd-bc41-4513-a069-bb3a66ec65af None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:02.867109 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-faf7aebd-bc41-4513-a069-bb3a66ec65af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:02.899303 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:49:02.924724 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf7aebd-bc41-4513-a069-bb3a66ec65af None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:02.928363 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d61c042-ca4b-42fb-984a-c590298707c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:02.929002 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61c042-ca4b-42fb-984a-c590298707c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:02.929286 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61c042-ca4b-42fb-984a-c590298707c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:02.929571 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61c042-ca4b-42fb-984a-c590298707c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:02.939964 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d61c042-ca4b-42fb-984a-c590298707c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:02.941411 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d61c042-ca4b-42fb-984a-c590298707c2 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:02.948900 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1224/2443] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:49:02 2022] POST /identity/v3/users => generated 289 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:49:02.949777 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62764b97-c69e-4487-848f-b61484488eaa admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:02.952426 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-62764b97-c69e-4487-848f-b61484488eaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:02.956188 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1220/2444] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:49:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:49:02.964581 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62764b97-c69e-4487-848f-b61484488eaa admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:02.969702 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462fc45c-8361-4ddd-83c5-d478a249de43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:02.970387 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462fc45c-8361-4ddd-83c5-d478a249de43 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:02.970667 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462fc45c-8361-4ddd-83c5-d478a249de43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:02.971014 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462fc45c-8361-4ddd-83c5-d478a249de43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:02.971712 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462fc45c-8361-4ddd-83c5-d478a249de43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:02.973299 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462fc45c-8361-4ddd-83c5-d478a249de43 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:02.984565 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1225/2445] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:49:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:49:02.988815 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d61c042-ca4b-42fb-984a-c590298707c2 service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:02.999312 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d61c042-ca4b-42fb-984a-c590298707c2 service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:03.004080 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91936b9-30b0-490d-9da5-fd763b4c9318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:03.004747 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91936b9-30b0-490d-9da5-fd763b4c9318 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:03.005055 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91936b9-30b0-490d-9da5-fd763b4c9318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:03.005323 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91936b9-30b0-490d-9da5-fd763b4c9318 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:03.006053 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91936b9-30b0-490d-9da5-fd763b4c9318 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:03.007770 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91936b9-30b0-490d-9da5-fd763b4c9318 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:03.018307 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1221/2446] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:49:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:49:03.022535 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462fc45c-8361-4ddd-83c5-d478a249de43 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:03.032985 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462fc45c-8361-4ddd-83c5-d478a249de43 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:03.038142 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bd666c-bd61-48ee-ac5f-0d48ecea8fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:03.039866 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd666c-bd61-48ee-ac5f-0d48ecea8fef admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:03.039866 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd666c-bd61-48ee-ac5f-0d48ecea8fef admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:03.039866 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd666c-bd61-48ee-ac5f-0d48ecea8fef admin admin] PATH_INFO: `/v3/projects/d1f57033b5c24fc4b80d1e60eba64fdb/users/71a6bf18f33c4c13b3ad1da3a2df12db/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:03.056869 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bd666c-bd61-48ee-ac5f-0d48ecea8fef admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1f57033b5c24fc4b80d1e60eba64fdb, user_id=71a6bf18f33c4c13b3ad1da3a2df12db, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:03.059622 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bd666c-bd61-48ee-ac5f-0d48ecea8fef admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:03.158935 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1226/2447] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:49:03 2022] PUT /identity/v3/projects/d1f57033b5c24fc4b80d1e60eba64fdb/users/71a6bf18f33c4c13b3ad1da3a2df12db/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:49:03.229818 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f50d0-8fa5-47b6-8dfc-b02f93018aa5 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:03.230156 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f50d0-8fa5-47b6-8dfc-b02f93018aa5 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:03.230383 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f50d0-8fa5-47b6-8dfc-b02f93018aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:03.266014 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-329f50d0-8fa5-47b6-8dfc-b02f93018aa5 None None] MFA Rules not processed for user `71a6bf18f33c4c13b3ad1da3a2df12db`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:49:03.267273 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-329f50d0-8fa5-47b6-8dfc-b02f93018aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:03.361595 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1222/2448] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:49:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:49:03.366270 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bd666c-bd61-48ee-ac5f-0d48ecea8fef admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:03.429024 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bd666c-bd61-48ee-ac5f-0d48ecea8fef admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:03.433677 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51ba32d-1a1f-4ccb-9269-60c0c5c253e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:03.434339 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ba32d-1a1f-4ccb-9269-60c0c5c253e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:03.436101 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ba32d-1a1f-4ccb-9269-60c0c5c253e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:03.436101 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ba32d-1a1f-4ccb-9269-60c0c5c253e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:03.436586 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51ba32d-1a1f-4ccb-9269-60c0c5c253e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:03.438511 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51ba32d-1a1f-4ccb-9269-60c0c5c253e0 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:03.462965 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329f50d0-8fa5-47b6-8dfc-b02f93018aa5 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:03.464425 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-329f50d0-8fa5-47b6-8dfc-b02f93018aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:03.479575 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329f50d0-8fa5-47b6-8dfc-b02f93018aa5 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:03.485439 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6646484e-d89e-4ad6-8952-a67232e36a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:03.486168 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6646484e-d89e-4ad6-8952-a67232e36a46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:03.486401 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6646484e-d89e-4ad6-8952-a67232e36a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:03.486682 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6646484e-d89e-4ad6-8952-a67232e36a46 admin admin] PATH_INFO: `/v3/projects/1e8ab187215e4413abdd1475bd615692` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:03.487973 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6646484e-d89e-4ad6-8952-a67232e36a46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e8ab187215e4413abdd1475bd615692)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:03.490665 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6646484e-d89e-4ad6-8952-a67232e36a46 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:03.525842 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1227/2449] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:49:03 2022] POST /identity/v3/projects => generated 377 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:49:03.530787 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51ba32d-1a1f-4ccb-9269-60c0c5c253e0 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:03.597142 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51ba32d-1a1f-4ccb-9269-60c0c5c253e0 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:03.611272 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-6646484e-d89e-4ad6-8952-a67232e36a46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e8ab187215e4413abdd1475bd615692', 'request_id': 'req-6646484e-d89e-4ad6-8952-a67232e36a46'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:03.616809 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-6646484e-d89e-4ad6-8952-a67232e36a46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e8ab187215e4413abdd1475bd615692', 'request_id': 'req-6646484e-d89e-4ad6-8952-a67232e36a46'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:03.623706 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-6646484e-d89e-4ad6-8952-a67232e36a46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e8ab187215e4413abdd1475bd615692', 'request_id': 'req-6646484e-d89e-4ad6-8952-a67232e36a46'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:03.624424 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:49:03.665473 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e70ed-9118-46b4-882e-830741f3f21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:03.666181 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e70ed-9118-46b4-882e-830741f3f21e admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:03.666462 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e70ed-9118-46b4-882e-830741f3f21e admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:03.666738 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e70ed-9118-46b4-882e-830741f3f21e admin admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:03.667434 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353e70ed-9118-46b4-882e-830741f3f21e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:03.669421 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353e70ed-9118-46b4-882e-830741f3f21e admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:03.689457 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-6646484e-d89e-4ad6-8952-a67232e36a46 admin admin] The token cache is being invalidate because project 1e8ab187215e4413abdd1475bd615692 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:49:03.689457 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-6646484e-d89e-4ad6-8952-a67232e36a46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:03.692992 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-6646484e-d89e-4ad6-8952-a67232e36a46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:03.692992 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1223/2450] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:49:03 2022] DELETE /identity/v3/projects/1e8ab187215e4413abdd1475bd615692 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:49:03.775990 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1228/2451] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:49:03 2022] POST /identity/v3/users => generated 289 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:49:03.780928 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6646484e-d89e-4ad6-8952-a67232e36a46 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:03.785332 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-6646484e-d89e-4ad6-8952-a67232e36a46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:03.800442 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6646484e-d89e-4ad6-8952-a67232e36a46 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:03.805528 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5621a7-b708-4860-8e73-de1c372e7c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:03.808281 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5621a7-b708-4860-8e73-de1c372e7c4e admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:03.808281 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5621a7-b708-4860-8e73-de1c372e7c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:03.808281 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5621a7-b708-4860-8e73-de1c372e7c4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:03.808281 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5621a7-b708-4860-8e73-de1c372e7c4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:03.809179 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5621a7-b708-4860-8e73-de1c372e7c4e admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:03.821879 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1224/2452] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:49:03.826310 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e70ed-9118-46b4-882e-830741f3f21e admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:03.844835 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e70ed-9118-46b4-882e-830741f3f21e admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:03.850997 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ea466d-6979-459d-9b50-a2f1ae163abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:03.852021 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea466d-6979-459d-9b50-a2f1ae163abf admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:03.852677 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea466d-6979-459d-9b50-a2f1ae163abf admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:03.852677 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea466d-6979-459d-9b50-a2f1ae163abf admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:03.853498 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ea466d-6979-459d-9b50-a2f1ae163abf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:03.855220 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ea466d-6979-459d-9b50-a2f1ae163abf admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:03.869514 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1229/2453] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:49:03.874617 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5621a7-b708-4860-8e73-de1c372e7c4e admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:03.887754 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5621a7-b708-4860-8e73-de1c372e7c4e admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:03.893902 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb98b414-9e30-426c-8791-5093090cbd72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:03.895198 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb98b414-9e30-426c-8791-5093090cbd72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:03.895826 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb98b414-9e30-426c-8791-5093090cbd72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:03.896259 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb98b414-9e30-426c-8791-5093090cbd72 admin admin] PATH_INFO: `/v3/projects/7d55503111b54bb78ef74202437db27a/users/ab4b99c39e144a27a8e125e56b7ead8a/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:03.917588 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb98b414-9e30-426c-8791-5093090cbd72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d55503111b54bb78ef74202437db27a, user_id=ab4b99c39e144a27a8e125e56b7ead8a, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:03.921019 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb98b414-9e30-426c-8791-5093090cbd72 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:04.038843 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:49:04.045121 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1225/2454] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:49:03 2022] PUT /identity/v3/projects/7d55503111b54bb78ef74202437db27a/users/ab4b99c39e144a27a8e125e56b7ead8a/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:49:04.060159 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32345b7c-8154-416a-93d9-abb0c151cab3 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:04.061410 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32345b7c-8154-416a-93d9-abb0c151cab3 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:04.061771 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32345b7c-8154-416a-93d9-abb0c151cab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:04.092145 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-32345b7c-8154-416a-93d9-abb0c151cab3 None None] MFA Rules not processed for user `ab4b99c39e144a27a8e125e56b7ead8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:49:04.093617 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-32345b7c-8154-416a-93d9-abb0c151cab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:04.163954 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1230/2455] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:49:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:49:04.169084 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb98b414-9e30-426c-8791-5093090cbd72 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:04.234868 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb98b414-9e30-426c-8791-5093090cbd72 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:04.240555 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d60852-92a0-4fdb-951c-55cf67bc3478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:04.241408 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d60852-92a0-4fdb-951c-55cf67bc3478 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:04.241737 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d60852-92a0-4fdb-951c-55cf67bc3478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:04.242055 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d60852-92a0-4fdb-951c-55cf67bc3478 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:04.242852 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d60852-92a0-4fdb-951c-55cf67bc3478 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:04.245049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d60852-92a0-4fdb-951c-55cf67bc3478 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:04.377745 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1226/2456] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:49:04 2022] POST /identity/v3/projects => generated 375 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:49:04.381238 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32345b7c-8154-416a-93d9-abb0c151cab3 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:04.393016 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32345b7c-8154-416a-93d9-abb0c151cab3 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:04.449787 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cee6845-00e6-4698-885f-c01daf081614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:04.450545 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cee6845-00e6-4698-885f-c01daf081614 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:04.450908 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cee6845-00e6-4698-885f-c01daf081614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:04.451195 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cee6845-00e6-4698-885f-c01daf081614 admin admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:04.451977 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cee6845-00e6-4698-885f-c01daf081614 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:04.453936 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cee6845-00e6-4698-885f-c01daf081614 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:04.555903 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1231/2457] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:49:04 2022] POST /identity/v3/users => generated 294 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:49:04.559835 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d60852-92a0-4fdb-951c-55cf67bc3478 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:04.571568 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d60852-92a0-4fdb-951c-55cf67bc3478 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:04.576516 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797fee8e-6e71-4521-ac68-6c94be850886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:04.577197 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797fee8e-6e71-4521-ac68-6c94be850886 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:04.577514 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797fee8e-6e71-4521-ac68-6c94be850886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:04.577922 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797fee8e-6e71-4521-ac68-6c94be850886 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:04.578663 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797fee8e-6e71-4521-ac68-6c94be850886 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:04.580608 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797fee8e-6e71-4521-ac68-6c94be850886 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:04.592320 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1227/2458] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:49:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:49:04.597428 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cee6845-00e6-4698-885f-c01daf081614 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:04.608994 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cee6845-00e6-4698-885f-c01daf081614 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:04.613837 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763ec3a3-592e-4203-88a0-b333df2b0696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:04.614734 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ec3a3-592e-4203-88a0-b333df2b0696 admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:04.615017 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ec3a3-592e-4203-88a0-b333df2b0696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:04.615291 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ec3a3-592e-4203-88a0-b333df2b0696 admin admin] PATH_INFO: `/v3/projects/2d4708690535408b870e1c1d51180aab/users/0065056bef144e318c3a6443b5444a13/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:04.632295 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763ec3a3-592e-4203-88a0-b333df2b0696 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d4708690535408b870e1c1d51180aab, user_id=0065056bef144e318c3a6443b5444a13, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:04.634962 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763ec3a3-592e-4203-88a0-b333df2b0696 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:04.694466 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:49:04.699200 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1232/2459] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:49:04 2022] PUT /identity/v3/projects/2d4708690535408b870e1c1d51180aab/users/0065056bef144e318c3a6443b5444a13/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:49:04.703425 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797fee8e-6e71-4521-ac68-6c94be850886 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:04.714018 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797fee8e-6e71-4521-ac68-6c94be850886 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:04.770669 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370b23c7-0ab1-4ca6-89d3-afe91f06a099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:04.771398 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b23c7-0ab1-4ca6-89d3-afe91f06a099 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:04.771719 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b23c7-0ab1-4ca6-89d3-afe91f06a099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:04.772003 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b23c7-0ab1-4ca6-89d3-afe91f06a099 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:04.772732 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370b23c7-0ab1-4ca6-89d3-afe91f06a099 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:04.774333 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370b23c7-0ab1-4ca6-89d3-afe91f06a099 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:04.784941 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1228/2460] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:49:04 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:49:04.789155 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763ec3a3-592e-4203-88a0-b333df2b0696 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:04.800353 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763ec3a3-592e-4203-88a0-b333df2b0696 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:04.805180 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d2c6f8-e54e-4e8c-998f-2439637e1f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:04.806173 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d2c6f8-e54e-4e8c-998f-2439637e1f7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:04.806398 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d2c6f8-e54e-4e8c-998f-2439637e1f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:04.806680 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d2c6f8-e54e-4e8c-998f-2439637e1f7b admin admin] PATH_INFO: `/v3/projects/2d4708690535408b870e1c1d51180aab/users/0065056bef144e318c3a6443b5444a13/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:04.808734 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d2c6f8-e54e-4e8c-998f-2439637e1f7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d4708690535408b870e1c1d51180aab, user_id=0065056bef144e318c3a6443b5444a13, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:04.811759 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d2c6f8-e54e-4e8c-998f-2439637e1f7b admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:04.822057 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1233/2461] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:49:04 2022] PUT /identity/v3/projects/2d4708690535408b870e1c1d51180aab/users/0065056bef144e318c3a6443b5444a13/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:49:04.833963 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736799f8-15c6-4e73-8d87-16d4720c9985 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:04.834276 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736799f8-15c6-4e73-8d87-16d4720c9985 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:04.834595 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736799f8-15c6-4e73-8d87-16d4720c9985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:04.862403 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-736799f8-15c6-4e73-8d87-16d4720c9985 None None] MFA Rules not processed for user `0065056bef144e318c3a6443b5444a13`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:49:04.863713 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-736799f8-15c6-4e73-8d87-16d4720c9985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:04.926451 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:49:04.936430 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1229/2462] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:49:04 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:49:04.947945 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d2c6f8-e54e-4e8c-998f-2439637e1f7b admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:04.950210 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-f0d2c6f8-e54e-4e8c-998f-2439637e1f7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:05.013864 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d2c6f8-e54e-4e8c-998f-2439637e1f7b admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:05.017603 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d15d645-a6e7-42a7-9923-2949cdcf1f7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:05.018265 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d15d645-a6e7-42a7-9923-2949cdcf1f7f service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:05.018633 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d15d645-a6e7-42a7-9923-2949cdcf1f7f service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:05.018825 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d15d645-a6e7-42a7-9923-2949cdcf1f7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:05.021133 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-4d15d645-a6e7-42a7-9923-2949cdcf1f7f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:05.089186 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d15d645-a6e7-42a7-9923-2949cdcf1f7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:05.090687 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d15d645-a6e7-42a7-9923-2949cdcf1f7f service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:05.101002 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1234/2463] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:49:04 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:49:07.032288 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736799f8-15c6-4e73-8d87-16d4720c9985 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:07.042521 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736799f8-15c6-4e73-8d87-16d4720c9985 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:07.046902 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7174cc33-9bdd-4220-9694-0624bf9f18f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:07.047596 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7174cc33-9bdd-4220-9694-0624bf9f18f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:07.047885 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7174cc33-9bdd-4220-9694-0624bf9f18f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:07.048163 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7174cc33-9bdd-4220-9694-0624bf9f18f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:07.106257 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7174cc33-9bdd-4220-9694-0624bf9f18f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:07.108388 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7174cc33-9bdd-4220-9694-0624bf9f18f8 service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:07.117963 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1230/2464] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:49:07 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:49:08.651097 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d15d645-a6e7-42a7-9923-2949cdcf1f7f service cinder] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:08.660545 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d15d645-a6e7-42a7-9923-2949cdcf1f7f service cinder] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:08.664332 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c8aac9-dc17-4a37-bdb1-ba6f4b5f5957 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:08.664997 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c8aac9-dc17-4a37-bdb1-ba6f4b5f5957 service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:08.665281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c8aac9-dc17-4a37-bdb1-ba6f4b5f5957 service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:08.665570 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c8aac9-dc17-4a37-bdb1-ba6f4b5f5957 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:08.676189 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c8aac9-dc17-4a37-bdb1-ba6f4b5f5957 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:08.677634 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c8aac9-dc17-4a37-bdb1-ba6f4b5f5957 service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:08.687563 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1235/2465] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:49:08 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:49:16.400443 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f2b052-273a-4837-b612-6d2757a1986f None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:16.400985 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f2b052-273a-4837-b612-6d2757a1986f None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:16.400985 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f2b052-273a-4837-b612-6d2757a1986f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:16.418797 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-58f2b052-273a-4837-b612-6d2757a1986f None None] MFA Rules not processed for user `da799b0044f54da6835aa06e43ce1a6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:49:16.421195 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-58f2b052-273a-4837-b612-6d2757a1986f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:16.485370 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1231/2466] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jul 18 16:49:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:49:16.552990 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c8aac9-dc17-4a37-bdb1-ba6f4b5f5957 service cinder] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:16.552990 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-87c8aac9-dc17-4a37-bdb1-ba6f4b5f5957 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:16.604368 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c8aac9-dc17-4a37-bdb1-ba6f4b5f5957 service cinder] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:16.610283 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf47da69-948e-40ae-9abf-1410fd385147 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43854d7a6df2466eb7976fabe6011767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:16.610942 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf47da69-948e-40ae-9abf-1410fd385147 service swift] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:16.611229 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf47da69-948e-40ae-9abf-1410fd385147 service swift] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:16.611588 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf47da69-948e-40ae-9abf-1410fd385147 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:16.620329 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf47da69-948e-40ae-9abf-1410fd385147 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:16.621914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf47da69-948e-40ae-9abf-1410fd385147 service swift] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:16.631085 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1236/2467] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jul 18 16:49:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 16:49:16.748802 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f2b052-273a-4837-b612-6d2757a1986f None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:16.833628 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f2b052-273a-4837-b612-6d2757a1986f None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:16.838875 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eb9157-c625-42e8-884f-e418a0033c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:16.840077 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb9157-c625-42e8-884f-e418a0033c77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:16.840379 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb9157-c625-42e8-884f-e418a0033c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:16.840665 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb9157-c625-42e8-884f-e418a0033c77 admin admin] PATH_INFO: `/v3/users/71a6bf18f33c4c13b3ad1da3a2df12db` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:16.843044 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eb9157-c625-42e8-884f-e418a0033c77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71a6bf18f33c4c13b3ad1da3a2df12db)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:16.845368 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eb9157-c625-42e8-884f-e418a0033c77 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:17.049614 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:49:17.058464 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-12eb9157-c625-42e8-884f-e418a0033c77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71a6bf18f33c4c13b3ad1da3a2df12db'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:17.132846 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-12eb9157-c625-42e8-884f-e418a0033c77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71a6bf18f33c4c13b3ad1da3a2df12db'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:17.145413 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-12eb9157-c625-42e8-884f-e418a0033c77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71a6bf18f33c4c13b3ad1da3a2df12db'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:17.163143 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-12eb9157-c625-42e8-884f-e418a0033c77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71a6bf18f33c4c13b3ad1da3a2df12db'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:17.164693 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-12eb9157-c625-42e8-884f-e418a0033c77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71a6bf18f33c4c13b3ad1da3a2df12db'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:17.176597 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91bba5d-7f58-4f0b-9acc-b7dd55d04699 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:17.176888 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91bba5d-7f58-4f0b-9acc-b7dd55d04699 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:17.177173 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91bba5d-7f58-4f0b-9acc-b7dd55d04699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:17.178143 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1232/2468] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:49:16 2022] DELETE /identity/v3/users/71a6bf18f33c4c13b3ad1da3a2df12db => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:49:17.199771 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-e91bba5d-7f58-4f0b-9acc-b7dd55d04699 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:49:17.201984 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-e91bba5d-7f58-4f0b-9acc-b7dd55d04699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:17.218902 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:49:17.274679 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1237/2469] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:49:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:49:17.287784 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eb9157-c625-42e8-884f-e418a0033c77 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:17.289659 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-12eb9157-c625-42e8-884f-e418a0033c77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:17.364361 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eb9157-c625-42e8-884f-e418a0033c77 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:17.368803 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55878c0-7b6a-4b21-8eb5-ff16716f2a73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:17.369754 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55878c0-7b6a-4b21-8eb5-ff16716f2a73 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:17.370157 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55878c0-7b6a-4b21-8eb5-ff16716f2a73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:17.370549 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55878c0-7b6a-4b21-8eb5-ff16716f2a73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:17.386060 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55878c0-7b6a-4b21-8eb5-ff16716f2a73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:17.387646 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55878c0-7b6a-4b21-8eb5-ff16716f2a73 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:17.400294 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1233/2470] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:49:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:49:18.078586 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91bba5d-7f58-4f0b-9acc-b7dd55d04699 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:18.080575 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-e91bba5d-7f58-4f0b-9acc-b7dd55d04699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:18.098372 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91bba5d-7f58-4f0b-9acc-b7dd55d04699 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:18.131815 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23c4f6a-fc91-478b-9141-955915ccebdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:18.135205 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c4f6a-fc91-478b-9141-955915ccebdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:18.138922 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c4f6a-fc91-478b-9141-955915ccebdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:18.151704 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c4f6a-fc91-478b-9141-955915ccebdb admin admin] PATH_INFO: `/v3/projects/d1f57033b5c24fc4b80d1e60eba64fdb` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:18.151704 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23c4f6a-fc91-478b-9141-955915ccebdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1f57033b5c24fc4b80d1e60eba64fdb)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:18.151704 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23c4f6a-fc91-478b-9141-955915ccebdb admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:18.277357 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-a23c4f6a-fc91-478b-9141-955915ccebdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1f57033b5c24fc4b80d1e60eba64fdb', 'request_id': 'req-a23c4f6a-fc91-478b-9141-955915ccebdb'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:18.285509 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-a23c4f6a-fc91-478b-9141-955915ccebdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1f57033b5c24fc4b80d1e60eba64fdb', 'request_id': 'req-a23c4f6a-fc91-478b-9141-955915ccebdb'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:18.376041 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-a23c4f6a-fc91-478b-9141-955915ccebdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1f57033b5c24fc4b80d1e60eba64fdb', 'request_id': 'req-a23c4f6a-fc91-478b-9141-955915ccebdb'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:18.388243 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-a23c4f6a-fc91-478b-9141-955915ccebdb admin admin] The token cache is being invalidate because project d1f57033b5c24fc4b80d1e60eba64fdb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:49:18.388674 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-a23c4f6a-fc91-478b-9141-955915ccebdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:18.389645 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-a23c4f6a-fc91-478b-9141-955915ccebdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:18.391350 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1238/2471] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:49:18 2022] DELETE /identity/v3/projects/d1f57033b5c24fc4b80d1e60eba64fdb => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:49:18.395348 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55878c0-7b6a-4b21-8eb5-ff16716f2a73 service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:18.406757 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55878c0-7b6a-4b21-8eb5-ff16716f2a73 service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:18.494953 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecdc0c3-4630-4b48-b7d3-67917cadf68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:18.496315 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecdc0c3-4630-4b48-b7d3-67917cadf68f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:18.496919 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecdc0c3-4630-4b48-b7d3-67917cadf68f admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:18.497529 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecdc0c3-4630-4b48-b7d3-67917cadf68f admin admin] PATH_INFO: `/v3/users/ab4b99c39e144a27a8e125e56b7ead8a` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:18.500343 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecdc0c3-4630-4b48-b7d3-67917cadf68f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab4b99c39e144a27a8e125e56b7ead8a)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:18.507033 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecdc0c3-4630-4b48-b7d3-67917cadf68f admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:18.774906 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-6ecdc0c3-4630-4b48-b7d3-67917cadf68f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab4b99c39e144a27a8e125e56b7ead8a'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:18.873307 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-6ecdc0c3-4630-4b48-b7d3-67917cadf68f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab4b99c39e144a27a8e125e56b7ead8a'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:18.879912 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:49:18.883335 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-6ecdc0c3-4630-4b48-b7d3-67917cadf68f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab4b99c39e144a27a8e125e56b7ead8a'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:18.897058 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-6ecdc0c3-4630-4b48-b7d3-67917cadf68f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab4b99c39e144a27a8e125e56b7ead8a'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:18.898113 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-6ecdc0c3-4630-4b48-b7d3-67917cadf68f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab4b99c39e144a27a8e125e56b7ead8a'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:18.905076 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1234/2472] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:49:18 2022] DELETE /identity/v3/users/ab4b99c39e144a27a8e125e56b7ead8a => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:49:19.718963 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23c4f6a-fc91-478b-9141-955915ccebdb admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:19.720457 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-a23c4f6a-fc91-478b-9141-955915ccebdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:19.733609 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23c4f6a-fc91-478b-9141-955915ccebdb admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:19.794813 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccefab61-ba9a-40cd-83ea-c4dca19db0f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:19.796134 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccefab61-ba9a-40cd-83ea-c4dca19db0f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:19.796418 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccefab61-ba9a-40cd-83ea-c4dca19db0f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:19.796696 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccefab61-ba9a-40cd-83ea-c4dca19db0f6 admin admin] PATH_INFO: `/v3/projects/7d55503111b54bb78ef74202437db27a` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:19.797910 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccefab61-ba9a-40cd-83ea-c4dca19db0f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d55503111b54bb78ef74202437db27a)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:19.799907 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccefab61-ba9a-40cd-83ea-c4dca19db0f6 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:19.959333 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:49:19.964333 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-ccefab61-ba9a-40cd-83ea-c4dca19db0f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d55503111b54bb78ef74202437db27a', 'request_id': 'req-ccefab61-ba9a-40cd-83ea-c4dca19db0f6'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:19.972302 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-ccefab61-ba9a-40cd-83ea-c4dca19db0f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d55503111b54bb78ef74202437db27a', 'request_id': 'req-ccefab61-ba9a-40cd-83ea-c4dca19db0f6'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:20.059313 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-ccefab61-ba9a-40cd-83ea-c4dca19db0f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d55503111b54bb78ef74202437db27a', 'request_id': 'req-ccefab61-ba9a-40cd-83ea-c4dca19db0f6'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:20.076414 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-ccefab61-ba9a-40cd-83ea-c4dca19db0f6 admin admin] The token cache is being invalidate because project 7d55503111b54bb78ef74202437db27a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:49:20.077940 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-ccefab61-ba9a-40cd-83ea-c4dca19db0f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:20.078211 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-ccefab61-ba9a-40cd-83ea-c4dca19db0f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:20.080197 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1239/2473] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:49:19 2022] DELETE /identity/v3/projects/7d55503111b54bb78ef74202437db27a => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:49:20.086011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecdc0c3-4630-4b48-b7d3-67917cadf68f admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:20.087764 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-6ecdc0c3-4630-4b48-b7d3-67917cadf68f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:20.108842 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecdc0c3-4630-4b48-b7d3-67917cadf68f admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:20.181693 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960a534c-0034-4925-97e0-c276510a1415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:20.183099 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960a534c-0034-4925-97e0-c276510a1415 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:20.183784 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960a534c-0034-4925-97e0-c276510a1415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:20.183784 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960a534c-0034-4925-97e0-c276510a1415 admin admin] PATH_INFO: `/v3/users/0065056bef144e318c3a6443b5444a13` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:20.185065 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960a534c-0034-4925-97e0-c276510a1415 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0065056bef144e318c3a6443b5444a13)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:20.189098 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960a534c-0034-4925-97e0-c276510a1415 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:20.441828 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-960a534c-0034-4925-97e0-c276510a1415 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0065056bef144e318c3a6443b5444a13'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:20.550650 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-960a534c-0034-4925-97e0-c276510a1415 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0065056bef144e318c3a6443b5444a13'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:20.560219 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-960a534c-0034-4925-97e0-c276510a1415 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0065056bef144e318c3a6443b5444a13'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:20.577354 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-960a534c-0034-4925-97e0-c276510a1415 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0065056bef144e318c3a6443b5444a13'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:20.578597 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-960a534c-0034-4925-97e0-c276510a1415 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0065056bef144e318c3a6443b5444a13'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:20.586200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1235/2474] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:49:20 2022] DELETE /identity/v3/users/0065056bef144e318c3a6443b5444a13 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:49:21.131724 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccefab61-ba9a-40cd-83ea-c4dca19db0f6 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:21.133861 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-ccefab61-ba9a-40cd-83ea-c4dca19db0f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:21.172172 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccefab61-ba9a-40cd-83ea-c4dca19db0f6 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:21.297681 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b20eea-0424-447e-b0b4-ebba80260b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:21.299233 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b20eea-0424-447e-b0b4-ebba80260b56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:21.299735 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b20eea-0424-447e-b0b4-ebba80260b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:21.300149 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b20eea-0424-447e-b0b4-ebba80260b56 admin admin] PATH_INFO: `/v3/projects/2d4708690535408b870e1c1d51180aab` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:21.302193 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b20eea-0424-447e-b0b4-ebba80260b56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d4708690535408b870e1c1d51180aab)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:21.305760 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b20eea-0424-447e-b0b4-ebba80260b56 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:21.475729 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-f4b20eea-0424-447e-b0b4-ebba80260b56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d4708690535408b870e1c1d51180aab', 'request_id': 'req-f4b20eea-0424-447e-b0b4-ebba80260b56'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:21.489211 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-f4b20eea-0424-447e-b0b4-ebba80260b56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d4708690535408b870e1c1d51180aab', 'request_id': 'req-f4b20eea-0424-447e-b0b4-ebba80260b56'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:21.549034 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-f4b20eea-0424-447e-b0b4-ebba80260b56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d4708690535408b870e1c1d51180aab', 'request_id': 'req-f4b20eea-0424-447e-b0b4-ebba80260b56'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:21.562943 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-f4b20eea-0424-447e-b0b4-ebba80260b56 admin admin] The token cache is being invalidate because project 2d4708690535408b870e1c1d51180aab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:49:21.563293 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-f4b20eea-0424-447e-b0b4-ebba80260b56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:21.563999 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-f4b20eea-0424-447e-b0b4-ebba80260b56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:21.566265 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1240/2475] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:49:21 2022] DELETE /identity/v3/projects/2d4708690535408b870e1c1d51180aab => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:49:21.570906 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327e6e08-0f08-4949-acad-5100486efd48 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:21.571788 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327e6e08-0f08-4949-acad-5100486efd48 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:21.571788 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327e6e08-0f08-4949-acad-5100486efd48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:21.594816 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-327e6e08-0f08-4949-acad-5100486efd48 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:49:21.596391 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-327e6e08-0f08-4949-acad-5100486efd48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:21.663454 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:49:21.675621 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1236/2476] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:49:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:49:21.683943 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d04e4b-b253-42f9-801b-bbeaba8a9302 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:21.683943 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d04e4b-b253-42f9-801b-bbeaba8a9302 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:21.684337 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d04e4b-b253-42f9-801b-bbeaba8a9302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:21.711141 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-c1d04e4b-b253-42f9-801b-bbeaba8a9302 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:49:21.713002 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-c1d04e4b-b253-42f9-801b-bbeaba8a9302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:21.728286 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1241/2477] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:49:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:49:21.729306 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327e6e08-0f08-4949-acad-5100486efd48 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:21.741292 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327e6e08-0f08-4949-acad-5100486efd48 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:21.746930 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d1985-4b63-4963-a2d0-60dd4d31dc29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:21.747674 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d1985-4b63-4963-a2d0-60dd4d31dc29 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:21.747965 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d1985-4b63-4963-a2d0-60dd4d31dc29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:21.748246 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d1985-4b63-4963-a2d0-60dd4d31dc29 admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:21.749167 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502d1985-4b63-4963-a2d0-60dd4d31dc29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:49:21.749486 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502d1985-4b63-4963-a2d0-60dd4d31dc29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:21.751207 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502d1985-4b63-4963-a2d0-60dd4d31dc29 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:21.770312 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1237/2478] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 16:49:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:49:21.771285 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d04e4b-b253-42f9-801b-bbeaba8a9302 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:21.786556 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d04e4b-b253-42f9-801b-bbeaba8a9302 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:21.791834 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd46fdc-db99-4885-a7c3-74f370cb48d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:21.792521 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd46fdc-db99-4885-a7c3-74f370cb48d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:21.792813 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd46fdc-db99-4885-a7c3-74f370cb48d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:21.793092 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd46fdc-db99-4885-a7c3-74f370cb48d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:21.793797 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd46fdc-db99-4885-a7c3-74f370cb48d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:21.796436 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd46fdc-db99-4885-a7c3-74f370cb48d5 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:21.915175 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1242/2479] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:49:21 2022] POST /identity/v3/projects => generated 363 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:49:21.920151 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d1985-4b63-4963-a2d0-60dd4d31dc29 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:21.932814 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d1985-4b63-4963-a2d0-60dd4d31dc29 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:22.013421 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e40383b-f42c-49d4-9080-dff846f918bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:22.014861 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e40383b-f42c-49d4-9080-dff846f918bd admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:22.015595 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e40383b-f42c-49d4-9080-dff846f918bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:22.015595 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e40383b-f42c-49d4-9080-dff846f918bd admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:22.016330 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e40383b-f42c-49d4-9080-dff846f918bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:22.018395 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e40383b-f42c-49d4-9080-dff846f918bd admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:22.151396 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1238/2480] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:49:21 2022] POST /identity/v3/users => generated 282 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:49:22.156756 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd46fdc-db99-4885-a7c3-74f370cb48d5 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:22.176925 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd46fdc-db99-4885-a7c3-74f370cb48d5 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:22.180934 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa74fc9e-e157-4b47-83ad-9b717186810c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:22.181794 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74fc9e-e157-4b47-83ad-9b717186810c admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:22.182131 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74fc9e-e157-4b47-83ad-9b717186810c admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:22.182536 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74fc9e-e157-4b47-83ad-9b717186810c admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:22.183393 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa74fc9e-e157-4b47-83ad-9b717186810c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:22.185269 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa74fc9e-e157-4b47-83ad-9b717186810c admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:22.205153 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1243/2481] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:49:22.207268 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e40383b-f42c-49d4-9080-dff846f918bd admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:22.224411 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e40383b-f42c-49d4-9080-dff846f918bd admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:22.231087 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befa0f89-0508-4332-8149-83526f3f88a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:22.232065 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa0f89-0508-4332-8149-83526f3f88a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:22.232655 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa0f89-0508-4332-8149-83526f3f88a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:22.233591 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa0f89-0508-4332-8149-83526f3f88a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:22.236486 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befa0f89-0508-4332-8149-83526f3f88a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:22.239257 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befa0f89-0508-4332-8149-83526f3f88a8 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:22.257194 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1239/2482] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:49:22.258464 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa74fc9e-e157-4b47-83ad-9b717186810c admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:22.270191 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa74fc9e-e157-4b47-83ad-9b717186810c admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:22.274912 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965c9c87-5dbf-417b-baad-fa7b0f478337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:22.275922 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c9c87-5dbf-417b-baad-fa7b0f478337 admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:22.276204 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c9c87-5dbf-417b-baad-fa7b0f478337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:22.276476 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c9c87-5dbf-417b-baad-fa7b0f478337 admin admin] PATH_INFO: `/v3/projects/aaeb16bbae374566926cf8512b00fc73/users/89def089e134402eb097529bcb112ca1/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:22.320631 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965c9c87-5dbf-417b-baad-fa7b0f478337 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaeb16bbae374566926cf8512b00fc73, user_id=89def089e134402eb097529bcb112ca1, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:22.324375 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965c9c87-5dbf-417b-baad-fa7b0f478337 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:22.456780 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:49:22.465225 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1244/2483] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:49:22 2022] PUT /identity/v3/projects/aaeb16bbae374566926cf8512b00fc73/users/89def089e134402eb097529bcb112ca1/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:49:22.477546 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4becb485-333d-4546-80e2-aa00fb4ccd5a None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:22.477884 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4becb485-333d-4546-80e2-aa00fb4ccd5a None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:22.478591 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4becb485-333d-4546-80e2-aa00fb4ccd5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:22.514334 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-4becb485-333d-4546-80e2-aa00fb4ccd5a None None] MFA Rules not processed for user `89def089e134402eb097529bcb112ca1`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:49:22.516790 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-4becb485-333d-4546-80e2-aa00fb4ccd5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:22.614927 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1240/2484] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:49:22 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:49:22.627827 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965c9c87-5dbf-417b-baad-fa7b0f478337 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:22.630011 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-965c9c87-5dbf-417b-baad-fa7b0f478337 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:22.707714 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965c9c87-5dbf-417b-baad-fa7b0f478337 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:22.711542 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d763391-e443-444e-87b8-43ce54b4c778 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:22.712271 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d763391-e443-444e-87b8-43ce54b4c778 service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:22.712518 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d763391-e443-444e-87b8-43ce54b4c778 service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:22.712857 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d763391-e443-444e-87b8-43ce54b4c778 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:22.723542 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d763391-e443-444e-87b8-43ce54b4c778 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:22.725125 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d763391-e443-444e-87b8-43ce54b4c778 service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:22.736931 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1245/2485] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:49:22 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:49:32.610875 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=73,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:49:32.730950 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=27 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:49:39.717074 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6efc8-8353-466b-9e01-1ed805e3be1a None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:39.717750 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6efc8-8353-466b-9e01-1ed805e3be1a None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:39.717750 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6efc8-8353-466b-9e01-1ed805e3be1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:39.737794 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-e3a6efc8-8353-466b-9e01-1ed805e3be1a None None] MFA Rules not processed for user `da799b0044f54da6835aa06e43ce1a6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:49:39.739001 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e3a6efc8-8353-466b-9e01-1ed805e3be1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:39.803070 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1241/2486] 10.222.0.26 () {58 vars in 967 bytes} [Mon Jul 18 16:49:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:49:39.816806 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d763391-e443-444e-87b8-43ce54b4c778 service cinder] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:39.818221 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-2d763391-e443-444e-87b8-43ce54b4c778 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:39.888358 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d763391-e443-444e-87b8-43ce54b4c778 service cinder] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:39.893427 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee230b7-5be2-4578-bb27-72e35be7c49f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43854d7a6df2466eb7976fabe6011767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:39.894097 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee230b7-5be2-4578-bb27-72e35be7c49f service swift] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:39.894368 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee230b7-5be2-4578-bb27-72e35be7c49f service swift] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:39.894647 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee230b7-5be2-4578-bb27-72e35be7c49f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:39.902853 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee230b7-5be2-4578-bb27-72e35be7c49f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:39.904506 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee230b7-5be2-4578-bb27-72e35be7c49f service swift] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:39.912876 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1246/2487] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jul 18 16:49:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 16:49:49.789430 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=25 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:49:49.909604 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=32 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:07.090668 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a6efc8-8353-466b-9e01-1ed805e3be1a None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:07.092229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e3a6efc8-8353-466b-9e01-1ed805e3be1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:07.108633 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a6efc8-8353-466b-9e01-1ed805e3be1a None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:07.113432 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78261de-2caf-4b2b-8822-a85a5b5f9469 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:07.114234 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78261de-2caf-4b2b-8822-a85a5b5f9469 service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:07.114567 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78261de-2caf-4b2b-8822-a85a5b5f9469 service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:07.115055 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78261de-2caf-4b2b-8822-a85a5b5f9469 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:07.118636 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e78261de-2caf-4b2b-8822-a85a5b5f9469 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:07.189198 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78261de-2caf-4b2b-8822-a85a5b5f9469 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:07.190685 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78261de-2caf-4b2b-8822-a85a5b5f9469 service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:07.203228 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1242/2488] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:51:07.467409 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee230b7-5be2-4578-bb27-72e35be7c49f service swift] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:07.469060 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-6ee230b7-5be2-4578-bb27-72e35be7c49f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:07.543338 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee230b7-5be2-4578-bb27-72e35be7c49f service swift] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:07.546898 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78261de-2caf-4b2b-8822-a85a5b5f9469 service cinder] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:07.547346 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8324f649-ebdd-4604-9411-a8dd74051e65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:07.548068 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324f649-ebdd-4604-9411-a8dd74051e65 service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:07.548335 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324f649-ebdd-4604-9411-a8dd74051e65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:07.548718 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324f649-ebdd-4604-9411-a8dd74051e65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:07.550451 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-8324f649-ebdd-4604-9411-a8dd74051e65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:07.557122 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78261de-2caf-4b2b-8822-a85a5b5f9469 service cinder] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:07.561015 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f787dd8-39b8-4d57-a25b-57aa0af38ff1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:07.561659 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f787dd8-39b8-4d57-a25b-57aa0af38ff1 service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:07.561940 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f787dd8-39b8-4d57-a25b-57aa0af38ff1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:07.562213 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f787dd8-39b8-4d57-a25b-57aa0af38ff1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:07.564196 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-2f787dd8-39b8-4d57-a25b-57aa0af38ff1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:07.630965 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f787dd8-39b8-4d57-a25b-57aa0af38ff1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:07.633071 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f787dd8-39b8-4d57-a25b-57aa0af38ff1 service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:07.633668 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8324f649-ebdd-4604-9411-a8dd74051e65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:07.635861 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8324f649-ebdd-4604-9411-a8dd74051e65 service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:07.644659 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1243/2489] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:07 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:51:07.652363 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1247/2490] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:51:08.853452 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f787dd8-39b8-4d57-a25b-57aa0af38ff1 service cinder] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:08.855068 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-2f787dd8-39b8-4d57-a25b-57aa0af38ff1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:08.868222 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f787dd8-39b8-4d57-a25b-57aa0af38ff1 service cinder] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:08.872839 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99a494c-96ea-4b3b-a193-27412aef2140 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:08.873262 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a494c-96ea-4b3b-a193-27412aef2140 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:08.873779 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a494c-96ea-4b3b-a193-27412aef2140 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:08.874126 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a494c-96ea-4b3b-a193-27412aef2140 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:08.883867 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-c99a494c-96ea-4b3b-a193-27412aef2140 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:08.901438 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99a494c-96ea-4b3b-a193-27412aef2140 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:08.904325 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99a494c-96ea-4b3b-a193-27412aef2140 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:08.922287 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1244/2491] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:51:17.643719 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=54 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:17.718709 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8324f649-ebdd-4604-9411-a8dd74051e65 service neutron] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:17.728802 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8324f649-ebdd-4604-9411-a8dd74051e65 service neutron] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:17.733065 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51d9500-6cda-4e0d-9f05-cbecf20bc629 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:17.733773 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d9500-6cda-4e0d-9f05-cbecf20bc629 service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:17.734058 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d9500-6cda-4e0d-9f05-cbecf20bc629 service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:17.734347 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d9500-6cda-4e0d-9f05-cbecf20bc629 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:17.736465 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-c51d9500-6cda-4e0d-9f05-cbecf20bc629 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:17.748755 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51d9500-6cda-4e0d-9f05-cbecf20bc629 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:17.753789 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51d9500-6cda-4e0d-9f05-cbecf20bc629 service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:17.761216 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1248/2492] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:51:18.916754 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=62 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:25.251879 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99a494c-96ea-4b3b-a193-27412aef2140 service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:25.255076 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-c99a494c-96ea-4b3b-a193-27412aef2140 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:25.274088 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99a494c-96ea-4b3b-a193-27412aef2140 service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:25.280804 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2755a6-3f21-4003-bf94-32be5c73cc5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:25.281549 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2755a6-3f21-4003-bf94-32be5c73cc5c service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:25.281900 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2755a6-3f21-4003-bf94-32be5c73cc5c service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:25.282274 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2755a6-3f21-4003-bf94-32be5c73cc5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:25.285693 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-4b2755a6-3f21-4003-bf94-32be5c73cc5c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:25.300549 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2755a6-3f21-4003-bf94-32be5c73cc5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:25.302629 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2755a6-3f21-4003-bf94-32be5c73cc5c service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:25.316327 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1245/2493] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:51:27.756293 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=6 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:35.309997 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=6 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:52:06.655533 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51d9500-6cda-4e0d-9f05-cbecf20bc629 service neutron] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:06.657361 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-c51d9500-6cda-4e0d-9f05-cbecf20bc629 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:06.703019 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51d9500-6cda-4e0d-9f05-cbecf20bc629 service neutron] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:06.709113 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b3f0e8-3da7-4caa-ba9a-ebc585c0ab77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:06.709786 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3f0e8-3da7-4caa-ba9a-ebc585c0ab77 service nova] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:06.710076 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3f0e8-3da7-4caa-ba9a-ebc585c0ab77 service nova] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:06.710358 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3f0e8-3da7-4caa-ba9a-ebc585c0ab77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:06.712530 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-00b3f0e8-3da7-4caa-ba9a-ebc585c0ab77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:06.781073 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b3f0e8-3da7-4caa-ba9a-ebc585c0ab77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:06.782686 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b3f0e8-3da7-4caa-ba9a-ebc585c0ab77 service nova] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:06.794611 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1249/2494] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:52:06 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:52:16.790298 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=39 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:53:20.307154 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2755a6-3f21-4003-bf94-32be5c73cc5c service nova] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:20.319739 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2755a6-3f21-4003-bf94-32be5c73cc5c service nova] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:20.323517 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39d69cf-92d7-42ac-8e5c-723edb1d4da9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:20.324155 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d69cf-92d7-42ac-8e5c-723edb1d4da9 service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:20.324434 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d69cf-92d7-42ac-8e5c-723edb1d4da9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:20.324708 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d69cf-92d7-42ac-8e5c-723edb1d4da9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:20.326654 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a39d69cf-92d7-42ac-8e5c-723edb1d4da9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:20.398565 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39d69cf-92d7-42ac-8e5c-723edb1d4da9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:20.400053 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39d69cf-92d7-42ac-8e5c-723edb1d4da9 service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:20.410008 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1246/2495] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:53:20 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:53:30.405083 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=30 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:54:27.163227 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b3f0e8-3da7-4caa-ba9a-ebc585c0ab77 service nova] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:54:27.200305 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b3f0e8-3da7-4caa-ba9a-ebc585c0ab77 service nova] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:54:27.204096 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b04336e-6900-4701-bdda-3c26cc6e180e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:54:27.204742 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04336e-6900-4701-bdda-3c26cc6e180e service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:54:27.205020 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04336e-6900-4701-bdda-3c26cc6e180e service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:54:27.205291 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04336e-6900-4701-bdda-3c26cc6e180e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:54:27.207065 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-9b04336e-6900-4701-bdda-3c26cc6e180e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:54:27.283195 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b04336e-6900-4701-bdda-3c26cc6e180e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:54:27.284725 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b04336e-6900-4701-bdda-3c26cc6e180e service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:54:27.295523 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1250/2496] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:54:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:54:27.323918 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39d69cf-92d7-42ac-8e5c-723edb1d4da9 service cinder] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:54:27.350567 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39d69cf-92d7-42ac-8e5c-723edb1d4da9 service cinder] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:54:27.355445 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f7599a-8d8b-4011-aa54-cf3c1eb99426 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:54:27.356124 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f7599a-8d8b-4011-aa54-cf3c1eb99426 service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:54:27.356403 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f7599a-8d8b-4011-aa54-cf3c1eb99426 service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:54:27.356673 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f7599a-8d8b-4011-aa54-cf3c1eb99426 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:54:27.365790 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f7599a-8d8b-4011-aa54-cf3c1eb99426 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:54:27.367859 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f7599a-8d8b-4011-aa54-cf3c1eb99426 service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:54:27.377700 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1247/2497] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:54:27 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:54:37.290894 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=42 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:54:37.373134 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=13 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:54:55.899890 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b04336e-6900-4701-bdda-3c26cc6e180e service cinder] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:54:55.901408 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-9b04336e-6900-4701-bdda-3c26cc6e180e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:54:56.026839 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b04336e-6900-4701-bdda-3c26cc6e180e service cinder] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:54:56.032429 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757f8738-4c6b-46f8-90f4-874786533f2d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70f704a231a5474e8067a6c2dd88b355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:54:56.033138 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757f8738-4c6b-46f8-90f4-874786533f2d service placement] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:54:56.033626 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757f8738-4c6b-46f8-90f4-874786533f2d service placement] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:54:56.033910 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757f8738-4c6b-46f8-90f4-874786533f2d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:54:56.037400 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-757f8738-4c6b-46f8-90f4-874786533f2d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:54:56.056654 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757f8738-4c6b-46f8-90f4-874786533f2d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:54:56.058280 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757f8738-4c6b-46f8-90f4-874786533f2d service placement] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:54:56.070070 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1251/2498] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:54:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:55:06.064867 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=41 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:56:08.818121 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f7599a-8d8b-4011-aa54-cf3c1eb99426 service cinder] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:08.847422 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f7599a-8d8b-4011-aa54-cf3c1eb99426 service cinder] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:08.851138 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99d0ccb-11cf-45ce-8b87-589a93687fe4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:08.851831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d0ccb-11cf-45ce-8b87-589a93687fe4 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:08.852120 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d0ccb-11cf-45ce-8b87-589a93687fe4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:08.852401 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d0ccb-11cf-45ce-8b87-589a93687fe4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:08.872322 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99d0ccb-11cf-45ce-8b87-589a93687fe4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:08.873786 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99d0ccb-11cf-45ce-8b87-589a93687fe4 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:08.897417 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1248/2499] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:56:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:56:10.086065 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757f8738-4c6b-46f8-90f4-874786533f2d service placement] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:10.096012 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757f8738-4c6b-46f8-90f4-874786533f2d service placement] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:10.099649 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58afa432-f4c8-40c9-a8a1-2fc2c1870e0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:10.100313 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58afa432-f4c8-40c9-a8a1-2fc2c1870e0b service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:10.100597 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58afa432-f4c8-40c9-a8a1-2fc2c1870e0b service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:10.100875 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58afa432-f4c8-40c9-a8a1-2fc2c1870e0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:10.112866 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58afa432-f4c8-40c9-a8a1-2fc2c1870e0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:10.114331 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58afa432-f4c8-40c9-a8a1-2fc2c1870e0b service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:10.126983 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1252/2500] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:56:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:56:18.894497 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=17 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:56:20.121418 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=6 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:56:27.686798 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99d0ccb-11cf-45ce-8b87-589a93687fe4 service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:27.699286 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99d0ccb-11cf-45ce-8b87-589a93687fe4 service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:27.703974 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dbaa30-b297-497c-9d26-db1fe318b433 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:27.704725 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dbaa30-b297-497c-9d26-db1fe318b433 service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:27.705109 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dbaa30-b297-497c-9d26-db1fe318b433 service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:27.705435 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dbaa30-b297-497c-9d26-db1fe318b433 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:27.715432 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dbaa30-b297-497c-9d26-db1fe318b433 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:27.716956 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dbaa30-b297-497c-9d26-db1fe318b433 service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:27.747159 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1249/2501] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:56:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:56:27.904491 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58afa432-f4c8-40c9-a8a1-2fc2c1870e0b service neutron] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:27.916177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58afa432-f4c8-40c9-a8a1-2fc2c1870e0b service neutron] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:27.919846 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3b44ea-8028-43b7-8a32-e78ce973fd49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:27.920507 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b44ea-8028-43b7-8a32-e78ce973fd49 service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:27.920900 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b44ea-8028-43b7-8a32-e78ce973fd49 service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:27.921796 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b44ea-8028-43b7-8a32-e78ce973fd49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:27.932912 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3b44ea-8028-43b7-8a32-e78ce973fd49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:27.934358 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3b44ea-8028-43b7-8a32-e78ce973fd49 service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:27.971581 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1253/2502] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:56:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:56:28.125769 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dbaa30-b297-497c-9d26-db1fe318b433 service cinder] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:28.137904 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dbaa30-b297-497c-9d26-db1fe318b433 service cinder] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:28.141637 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3d4f76-ac5d-49ab-8fd5-6c3cfc004219 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:28.142990 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d4f76-ac5d-49ab-8fd5-6c3cfc004219 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:28.143110 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d4f76-ac5d-49ab-8fd5-6c3cfc004219 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:28.143400 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d4f76-ac5d-49ab-8fd5-6c3cfc004219 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:28.145249 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-4f3d4f76-ac5d-49ab-8fd5-6c3cfc004219 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:56:28.274478 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3d4f76-ac5d-49ab-8fd5-6c3cfc004219 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:28.276004 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3d4f76-ac5d-49ab-8fd5-6c3cfc004219 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:28.297400 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1250/2503] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:56:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:56:29.109749 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3b44ea-8028-43b7-8a32-e78ce973fd49 service cinder] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:29.148514 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3b44ea-8028-43b7-8a32-e78ce973fd49 service cinder] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:29.153964 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d07ce3-1ddb-4b5a-a9c6-00bf98aa0e5e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70f704a231a5474e8067a6c2dd88b355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:29.154716 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d07ce3-1ddb-4b5a-a9c6-00bf98aa0e5e service placement] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:29.154994 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d07ce3-1ddb-4b5a-a9c6-00bf98aa0e5e service placement] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:29.155272 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d07ce3-1ddb-4b5a-a9c6-00bf98aa0e5e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:29.158549 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-f5d07ce3-1ddb-4b5a-a9c6-00bf98aa0e5e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:56:29.179703 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d07ce3-1ddb-4b5a-a9c6-00bf98aa0e5e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:29.182549 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d07ce3-1ddb-4b5a-a9c6-00bf98aa0e5e service placement] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:29.196263 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1254/2504] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:56:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:56:29.291307 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3d4f76-ac5d-49ab-8fd5-6c3cfc004219 service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:29.293673 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-4f3d4f76-ac5d-49ab-8fd5-6c3cfc004219 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:56:29.321616 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3d4f76-ac5d-49ab-8fd5-6c3cfc004219 service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:29.330617 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a62ab10-3cee-498c-b174-5bb093f70e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:29.332185 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62ab10-3cee-498c-b174-5bb093f70e5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:29.332688 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62ab10-3cee-498c-b174-5bb093f70e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:29.340636 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62ab10-3cee-498c-b174-5bb093f70e5a admin admin] PATH_INFO: `/v3/users/e548ea934bc14b7e921f7e161c5496cf` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:29.340636 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a62ab10-3cee-498c-b174-5bb093f70e5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e548ea934bc14b7e921f7e161c5496cf)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:29.340636 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a62ab10-3cee-498c-b174-5bb093f70e5a admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:29.585706 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-1a62ab10-3cee-498c-b174-5bb093f70e5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e548ea934bc14b7e921f7e161c5496cf'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:29.677911 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-1a62ab10-3cee-498c-b174-5bb093f70e5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e548ea934bc14b7e921f7e161c5496cf'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:29.685599 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-1a62ab10-3cee-498c-b174-5bb093f70e5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e548ea934bc14b7e921f7e161c5496cf'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:29.698569 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-1a62ab10-3cee-498c-b174-5bb093f70e5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e548ea934bc14b7e921f7e161c5496cf'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:29.699321 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-1a62ab10-3cee-498c-b174-5bb093f70e5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e548ea934bc14b7e921f7e161c5496cf'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:29.705529 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1251/2505] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 16:56:29 2022] DELETE /identity/v3/users/e548ea934bc14b7e921f7e161c5496cf => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:56:29.967904 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d07ce3-1ddb-4b5a-a9c6-00bf98aa0e5e service placement] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:29.979561 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d07ce3-1ddb-4b5a-a9c6-00bf98aa0e5e service placement] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:30.052798 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e87ab4-8b01-4984-9150-3b5d9768bd01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:30.053665 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e87ab4-8b01-4984-9150-3b5d9768bd01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:30.053951 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e87ab4-8b01-4984-9150-3b5d9768bd01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:30.054228 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e87ab4-8b01-4984-9150-3b5d9768bd01 admin admin] PATH_INFO: `/v3/projects/b2153f74cf4b4bdcb0aab5b6921ca3ad` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:30.055464 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e87ab4-8b01-4984-9150-3b5d9768bd01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2153f74cf4b4bdcb0aab5b6921ca3ad)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:30.057430 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e87ab4-8b01-4984-9150-3b5d9768bd01 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:30.217290 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-a8e87ab4-8b01-4984-9150-3b5d9768bd01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2153f74cf4b4bdcb0aab5b6921ca3ad', 'request_id': 'req-a8e87ab4-8b01-4984-9150-3b5d9768bd01'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:30.224966 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-a8e87ab4-8b01-4984-9150-3b5d9768bd01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2153f74cf4b4bdcb0aab5b6921ca3ad', 'request_id': 'req-a8e87ab4-8b01-4984-9150-3b5d9768bd01'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:30.300823 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-a8e87ab4-8b01-4984-9150-3b5d9768bd01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2153f74cf4b4bdcb0aab5b6921ca3ad', 'request_id': 'req-a8e87ab4-8b01-4984-9150-3b5d9768bd01'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:30.313988 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-a8e87ab4-8b01-4984-9150-3b5d9768bd01 admin admin] The token cache is being invalidate because project b2153f74cf4b4bdcb0aab5b6921ca3ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:56:30.314388 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-a8e87ab4-8b01-4984-9150-3b5d9768bd01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:30.315153 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-a8e87ab4-8b01-4984-9150-3b5d9768bd01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:30.316785 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1255/2506] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 16:56:29 2022] DELETE /identity/v3/projects/b2153f74cf4b4bdcb0aab5b6921ca3ad => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:56:30.322426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b047e5-6498-45b9-ba51-69dddf093ca7 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:30.322917 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b047e5-6498-45b9-ba51-69dddf093ca7 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:30.323012 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b047e5-6498-45b9-ba51-69dddf093ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:30.334161 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:56:30.381896 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-39b047e5-6498-45b9-ba51-69dddf093ca7 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:56:30.383134 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-39b047e5-6498-45b9-ba51-69dddf093ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:56:30.447299 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1252/2507] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:56:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:56:30.456144 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d626c34-345e-4b68-8223-3c510b432a61 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:30.456434 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d626c34-345e-4b68-8223-3c510b432a61 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:30.456712 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d626c34-345e-4b68-8223-3c510b432a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:30.476181 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-3d626c34-345e-4b68-8223-3c510b432a61 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:56:30.477410 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-3d626c34-345e-4b68-8223-3c510b432a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:56:30.485818 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1256/2508] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:56:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:56:30.491108 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b047e5-6498-45b9-ba51-69dddf093ca7 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:30.501921 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b047e5-6498-45b9-ba51-69dddf093ca7 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:30.508116 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cb0c3a-1a9f-417b-b6e3-c1136ca91ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:30.508793 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb0c3a-1a9f-417b-b6e3-c1136ca91ecc admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:30.509079 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb0c3a-1a9f-417b-b6e3-c1136ca91ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:30.509357 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb0c3a-1a9f-417b-b6e3-c1136ca91ecc admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:30.510099 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cb0c3a-1a9f-417b-b6e3-c1136ca91ecc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:56:30.510410 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cb0c3a-1a9f-417b-b6e3-c1136ca91ecc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:30.512085 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cb0c3a-1a9f-417b-b6e3-c1136ca91ecc admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:30.525339 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1253/2509] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 16:56:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:56:30.529866 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d626c34-345e-4b68-8223-3c510b432a61 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:30.540299 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d626c34-345e-4b68-8223-3c510b432a61 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:30.545042 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97911e3-bc93-4a97-bacc-1228ea2ced5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:30.545704 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97911e3-bc93-4a97-bacc-1228ea2ced5a admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:30.546015 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97911e3-bc93-4a97-bacc-1228ea2ced5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:30.546321 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97911e3-bc93-4a97-bacc-1228ea2ced5a admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:30.547015 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97911e3-bc93-4a97-bacc-1228ea2ced5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:30.549969 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97911e3-bc93-4a97-bacc-1228ea2ced5a admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:30.675132 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1257/2510] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 16:56:30 2022] POST /identity/v3/projects => generated 397 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:56:30.680000 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cb0c3a-1a9f-417b-b6e3-c1136ca91ecc admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:30.691304 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cb0c3a-1a9f-417b-b6e3-c1136ca91ecc admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:30.752499 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea135e3-8e2c-44c3-9bb6-9f30b6af5451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:30.753181 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea135e3-8e2c-44c3-9bb6-9f30b6af5451 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:30.753469 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea135e3-8e2c-44c3-9bb6-9f30b6af5451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:30.753756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea135e3-8e2c-44c3-9bb6-9f30b6af5451 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:30.754517 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea135e3-8e2c-44c3-9bb6-9f30b6af5451 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:30.756544 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea135e3-8e2c-44c3-9bb6-9f30b6af5451 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:30.880706 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1254/2511] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 16:56:30 2022] POST /identity/v3/users => generated 299 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:56:30.885031 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97911e3-bc93-4a97-bacc-1228ea2ced5a admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:30.896390 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97911e3-bc93-4a97-bacc-1228ea2ced5a admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:30.901191 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ebf4b7-1d08-4810-a9c1-08402491c1fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:30.901858 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ebf4b7-1d08-4810-a9c1-08402491c1fa admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:30.902136 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ebf4b7-1d08-4810-a9c1-08402491c1fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:30.902407 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ebf4b7-1d08-4810-a9c1-08402491c1fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:30.903167 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ebf4b7-1d08-4810-a9c1-08402491c1fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:30.904791 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ebf4b7-1d08-4810-a9c1-08402491c1fa admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:30.910882 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:56:30.915955 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1258/2512] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:56:30.920343 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea135e3-8e2c-44c3-9bb6-9f30b6af5451 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:30.932081 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea135e3-8e2c-44c3-9bb6-9f30b6af5451 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:30.936943 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2764797b-a6ac-4ff8-b6e8-3cb5713ae6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:30.937615 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2764797b-a6ac-4ff8-b6e8-3cb5713ae6b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:30.937904 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2764797b-a6ac-4ff8-b6e8-3cb5713ae6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:30.938184 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2764797b-a6ac-4ff8-b6e8-3cb5713ae6b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:30.939734 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2764797b-a6ac-4ff8-b6e8-3cb5713ae6b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:30.941355 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2764797b-a6ac-4ff8-b6e8-3cb5713ae6b9 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:30.952719 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1255/2513] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 16:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:56:30.957734 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ebf4b7-1d08-4810-a9c1-08402491c1fa admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:30.968588 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ebf4b7-1d08-4810-a9c1-08402491c1fa admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:30.974180 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dfba73-e9b8-4bef-ad41-40762f49fa1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:30.975083 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dfba73-e9b8-4bef-ad41-40762f49fa1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:30.975365 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dfba73-e9b8-4bef-ad41-40762f49fa1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:30.975685 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dfba73-e9b8-4bef-ad41-40762f49fa1a admin admin] PATH_INFO: `/v3/projects/0396828d8a6c423c85c3105cf2776760/users/c7c3d91cd21744548a085f2339a08812/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:30.992327 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dfba73-e9b8-4bef-ad41-40762f49fa1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0396828d8a6c423c85c3105cf2776760, user_id=c7c3d91cd21744548a085f2339a08812, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:30.995361 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dfba73-e9b8-4bef-ad41-40762f49fa1a admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:31.078034 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1259/2514] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 16:56:30 2022] PUT /identity/v3/projects/0396828d8a6c423c85c3105cf2776760/users/c7c3d91cd21744548a085f2339a08812/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:56:31.145021 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddee231-8059-4939-bf30-1af678f6eba8 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:31.145302 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddee231-8059-4939-bf30-1af678f6eba8 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:31.145587 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddee231-8059-4939-bf30-1af678f6eba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:31.174814 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-9ddee231-8059-4939-bf30-1af678f6eba8 None None] MFA Rules not processed for user `c7c3d91cd21744548a085f2339a08812`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:56:31.176228 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-9ddee231-8059-4939-bf30-1af678f6eba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:56:31.211886 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:56:31.257552 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1256/2515] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 16:56:31 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:56:41.107515 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=10,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:56:41.296331 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=12 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:56:57.595547 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dfba73-e9b8-4bef-ad41-40762f49fa1a admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:57.597308 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-d4dfba73-e9b8-4bef-ad41-40762f49fa1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:56:57.670418 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dfba73-e9b8-4bef-ad41-40762f49fa1a admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:57.675026 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ca4be4-f5e3-4f88-9365-a262b0ee87c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:57.675707 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca4be4-f5e3-4f88-9365-a262b0ee87c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:57.676048 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca4be4-f5e3-4f88-9365-a262b0ee87c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:57.676325 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca4be4-f5e3-4f88-9365-a262b0ee87c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:57.678142 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-70ca4be4-f5e3-4f88-9365-a262b0ee87c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:56:57.687747 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ca4be4-f5e3-4f88-9365-a262b0ee87c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:57.689313 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ca4be4-f5e3-4f88-9365-a262b0ee87c7 service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:57.697296 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1260/2516] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:56:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:57:06.560224 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddee231-8059-4939-bf30-1af678f6eba8 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:57:06.561687 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-9ddee231-8059-4939-bf30-1af678f6eba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:57:06.650596 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddee231-8059-4939-bf30-1af678f6eba8 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:57:06.656385 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d624ae-c33b-4856-91c6-de82e897f530 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:57:06.657551 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d624ae-c33b-4856-91c6-de82e897f530 service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:57:06.657835 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d624ae-c33b-4856-91c6-de82e897f530 service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:57:06.658109 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d624ae-c33b-4856-91c6-de82e897f530 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:57:06.659958 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-66d624ae-c33b-4856-91c6-de82e897f530 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:57:06.740603 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d624ae-c33b-4856-91c6-de82e897f530 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:57:06.742468 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d624ae-c33b-4856-91c6-de82e897f530 service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:57:06.760766 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1257/2517] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:57:06 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:57:07.693755 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=31 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:57:09.781919 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ca4be4-f5e3-4f88-9365-a262b0ee87c7 service neutron] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:57:09.795667 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ca4be4-f5e3-4f88-9365-a262b0ee87c7 service neutron] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:57:09.802235 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b016c61c-2b77-426d-b962-c0377b776538 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:57:09.803063 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b016c61c-2b77-426d-b962-c0377b776538 service nova] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:57:09.803437 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b016c61c-2b77-426d-b962-c0377b776538 service nova] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:57:09.803821 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b016c61c-2b77-426d-b962-c0377b776538 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:57:09.806145 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-b016c61c-2b77-426d-b962-c0377b776538 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:57:09.822068 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b016c61c-2b77-426d-b962-c0377b776538 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:57:09.823793 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b016c61c-2b77-426d-b962-c0377b776538 service nova] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:57:09.842391 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1261/2518] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:57:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:57:16.758159 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=74 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:57:19.836131 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=6 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:57:21.035652 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d624ae-c33b-4856-91c6-de82e897f530 service nova] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:57:21.037101 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-66d624ae-c33b-4856-91c6-de82e897f530 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:57:21.117266 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d624ae-c33b-4856-91c6-de82e897f530 service nova] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:57:21.121021 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c229f2a3-ca11-4522-8d9a-12d7b23b3995 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:57:21.121764 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229f2a3-ca11-4522-8d9a-12d7b23b3995 service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:57:21.122048 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229f2a3-ca11-4522-8d9a-12d7b23b3995 service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:57:21.122369 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229f2a3-ca11-4522-8d9a-12d7b23b3995 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:57:21.132348 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c229f2a3-ca11-4522-8d9a-12d7b23b3995 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:57:21.133734 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c229f2a3-ca11-4522-8d9a-12d7b23b3995 service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:57:21.146392 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1258/2519] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:57:21 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:57:31.140402 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=31 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:59:28.324897 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b016c61c-2b77-426d-b962-c0377b776538 service nova] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:59:28.337138 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b016c61c-2b77-426d-b962-c0377b776538 service nova] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:59:28.342757 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17f4fbc-b606-4923-a72d-9b3e312de290 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:59:28.343444 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f4fbc-b606-4923-a72d-9b3e312de290 service nova] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:59:28.343763 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f4fbc-b606-4923-a72d-9b3e312de290 service nova] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:59:28.344042 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f4fbc-b606-4923-a72d-9b3e312de290 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:59:28.345829 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-c17f4fbc-b606-4923-a72d-9b3e312de290 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:59:28.427853 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17f4fbc-b606-4923-a72d-9b3e312de290 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:59:28.429567 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17f4fbc-b606-4923-a72d-9b3e312de290 service nova] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:59:28.448736 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1262/2520] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 16:59:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:59:38.445916 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=41 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:00:00.897205 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c229f2a3-ca11-4522-8d9a-12d7b23b3995 service cinder] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:00:00.899058 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-c229f2a3-ca11-4522-8d9a-12d7b23b3995 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:00:00.972245 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c229f2a3-ca11-4522-8d9a-12d7b23b3995 service cinder] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:00:00.978126 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2d5df-65e4-447f-b895-9bc38184a523 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70f704a231a5474e8067a6c2dd88b355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:00:00.978772 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2d5df-65e4-447f-b895-9bc38184a523 service placement] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:00:00.979057 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2d5df-65e4-447f-b895-9bc38184a523 service placement] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:00:00.979336 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2d5df-65e4-447f-b895-9bc38184a523 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:00:00.981167 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-edc2d5df-65e4-447f-b895-9bc38184a523 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:00:00.993457 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc2d5df-65e4-447f-b895-9bc38184a523 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:00:00.995049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc2d5df-65e4-447f-b895-9bc38184a523 service placement] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:00:01.005289 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1259/2521] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:00:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:00:11.001038 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=32 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:00:30.160286 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17f4fbc-b606-4923-a72d-9b3e312de290 service nova] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:00:30.160286 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-c17f4fbc-b606-4923-a72d-9b3e312de290 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:00:30.177610 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17f4fbc-b606-4923-a72d-9b3e312de290 service nova] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:00:30.182235 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1111075-8d86-45af-8d51-9b859b590a74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:00:30.183085 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1111075-8d86-45af-8d51-9b859b590a74 service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:00:30.183370 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1111075-8d86-45af-8d51-9b859b590a74 service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:00:30.183694 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1111075-8d86-45af-8d51-9b859b590a74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:00:30.186399 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-c1111075-8d86-45af-8d51-9b859b590a74 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:00:30.297347 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1111075-8d86-45af-8d51-9b859b590a74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:00:30.298769 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1111075-8d86-45af-8d51-9b859b590a74 service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:00:30.327573 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1263/2522] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:00:30 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:00:40.324493 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=41 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:01:22.197183 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2d5df-65e4-447f-b895-9bc38184a523 service placement] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:01:22.208755 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2d5df-65e4-447f-b895-9bc38184a523 service placement] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:01:22.212698 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b8145c-6725-4323-b856-190c423cf7cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:01:22.213382 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8145c-6725-4323-b856-190c423cf7cd service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:01:22.213644 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8145c-6725-4323-b856-190c423cf7cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:01:22.213917 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8145c-6725-4323-b856-190c423cf7cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:01:22.216938 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-55b8145c-6725-4323-b856-190c423cf7cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:01:22.239114 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b8145c-6725-4323-b856-190c423cf7cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:01:22.240848 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b8145c-6725-4323-b856-190c423cf7cd service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:01:22.254758 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1260/2523] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:01:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:01:22.368361 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1111075-8d86-45af-8d51-9b859b590a74 service cinder] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:01:22.392178 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1111075-8d86-45af-8d51-9b859b590a74 service cinder] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:01:22.396488 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86a8232-92e5-424d-865f-00e1fc3af08c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:01:22.396488 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a8232-92e5-424d-865f-00e1fc3af08c service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:01:22.396488 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a8232-92e5-424d-865f-00e1fc3af08c service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:01:22.397131 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a8232-92e5-424d-865f-00e1fc3af08c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:01:22.401746 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-e86a8232-92e5-424d-865f-00e1fc3af08c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:01:22.456382 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86a8232-92e5-424d-865f-00e1fc3af08c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:01:22.466696 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86a8232-92e5-424d-865f-00e1fc3af08c service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:01:22.495358 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1264/2524] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:01:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:01:22.874231 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b8145c-6725-4323-b856-190c423cf7cd service cinder] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:01:22.876922 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-55b8145c-6725-4323-b856-190c423cf7cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:01:22.898644 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b8145c-6725-4323-b856-190c423cf7cd service cinder] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:01:22.908076 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fb4154-594e-4f5e-b536-ec3fc38969fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:01:22.908810 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fb4154-594e-4f5e-b536-ec3fc38969fe service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:01:22.913054 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fb4154-594e-4f5e-b536-ec3fc38969fe service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:01:22.913054 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fb4154-594e-4f5e-b536-ec3fc38969fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:01:22.913054 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-56fb4154-594e-4f5e-b536-ec3fc38969fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:01:22.927797 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fb4154-594e-4f5e-b536-ec3fc38969fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:01:22.929459 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fb4154-594e-4f5e-b536-ec3fc38969fe service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:01:22.942536 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1261/2525] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:01:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:01:27.125904 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86a8232-92e5-424d-865f-00e1fc3af08c service cinder] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:01:27.138374 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86a8232-92e5-424d-865f-00e1fc3af08c service cinder] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:01:27.144840 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c02f2a6-8555-419d-be93-7e0de5e470a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:01:27.145547 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c02f2a6-8555-419d-be93-7e0de5e470a2 service nova] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:01:27.145883 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c02f2a6-8555-419d-be93-7e0de5e470a2 service nova] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:01:27.146188 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c02f2a6-8555-419d-be93-7e0de5e470a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:01:27.148070 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-1c02f2a6-8555-419d-be93-7e0de5e470a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:01:27.231741 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c02f2a6-8555-419d-be93-7e0de5e470a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:01:27.233496 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c02f2a6-8555-419d-be93-7e0de5e470a2 service nova] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:01:27.244306 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1265/2526] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:01:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:01:28.704676 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fb4154-594e-4f5e-b536-ec3fc38969fe service nova] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:01:28.717639 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fb4154-594e-4f5e-b536-ec3fc38969fe service nova] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:01:28.723894 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dbeef6-8f25-40b2-8661-d0cec08a318c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:01:28.724575 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dbeef6-8f25-40b2-8661-d0cec08a318c service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:01:28.724864 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dbeef6-8f25-40b2-8661-d0cec08a318c service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:01:28.725146 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dbeef6-8f25-40b2-8661-d0cec08a318c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:01:28.727151 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-71dbeef6-8f25-40b2-8661-d0cec08a318c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:01:28.807749 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dbeef6-8f25-40b2-8661-d0cec08a318c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:01:28.809956 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dbeef6-8f25-40b2-8661-d0cec08a318c service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:01:28.820948 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1262/2527] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:01:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:01:37.239345 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=39 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:01:38.815888 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=45 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:02:03.606150 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c02f2a6-8555-419d-be93-7e0de5e470a2 service nova] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:03.617461 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c02f2a6-8555-419d-be93-7e0de5e470a2 service nova] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:03.621629 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387e690c-1fb2-4c47-a4ab-fe275c815711 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:03.623228 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e690c-1fb2-4c47-a4ab-fe275c815711 service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:03.623610 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e690c-1fb2-4c47-a4ab-fe275c815711 service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:03.623971 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e690c-1fb2-4c47-a4ab-fe275c815711 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:03.633496 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387e690c-1fb2-4c47-a4ab-fe275c815711 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:03.634941 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387e690c-1fb2-4c47-a4ab-fe275c815711 service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:03.647530 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1266/2528] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:02:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:02:05.734332 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dbeef6-8f25-40b2-8661-d0cec08a318c service nova] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:05.745770 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dbeef6-8f25-40b2-8661-d0cec08a318c service nova] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:05.752078 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c271f0-0e03-4516-a7b4-30a5bb8db29a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:05.752725 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c271f0-0e03-4516-a7b4-30a5bb8db29a service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:05.753015 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c271f0-0e03-4516-a7b4-30a5bb8db29a service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:05.753296 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c271f0-0e03-4516-a7b4-30a5bb8db29a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:05.761652 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c271f0-0e03-4516-a7b4-30a5bb8db29a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:05.763925 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c271f0-0e03-4516-a7b4-30a5bb8db29a service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:05.773846 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1263/2529] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:02:05 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:02:13.640771 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=6 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:02:15.769128 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=6 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:02:30.551525 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387e690c-1fb2-4c47-a4ab-fe275c815711 service neutron] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:30.635308 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387e690c-1fb2-4c47-a4ab-fe275c815711 service neutron] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:30.644095 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3807c298-d1c1-41fd-934d-82821a77ac36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:30.644912 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3807c298-d1c1-41fd-934d-82821a77ac36 service nova] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:30.645201 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3807c298-d1c1-41fd-934d-82821a77ac36 service nova] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:30.645649 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3807c298-d1c1-41fd-934d-82821a77ac36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:30.657732 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c271f0-0e03-4516-a7b4-30a5bb8db29a service nova] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:30.662200 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3807c298-d1c1-41fd-934d-82821a77ac36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:30.663874 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3807c298-d1c1-41fd-934d-82821a77ac36 service nova] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:30.667682 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c271f0-0e03-4516-a7b4-30a5bb8db29a service nova] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:30.671685 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef7f9d6-6ecd-4ae6-a5f1-6ee047e6a61e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:30.672264 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7f9d6-6ecd-4ae6-a5f1-6ee047e6a61e service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:30.673669 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7f9d6-6ecd-4ae6-a5f1-6ee047e6a61e service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:30.673950 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7f9d6-6ecd-4ae6-a5f1-6ee047e6a61e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:30.676708 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1267/2530] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:02:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:02:30.684115 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef7f9d6-6ecd-4ae6-a5f1-6ee047e6a61e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:30.685625 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef7f9d6-6ecd-4ae6-a5f1-6ee047e6a61e service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:30.696016 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1264/2531] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:02:30 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 17:02:40.671485 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=15 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:02:40.691621 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=6 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:03:11.417289 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3807c298-d1c1-41fd-934d-82821a77ac36 service nova] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:03:11.427868 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3807c298-d1c1-41fd-934d-82821a77ac36 service nova] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:03:11.431391 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357abe0c-7e50-41bb-8ea1-4dd262109fb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:03:11.432071 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357abe0c-7e50-41bb-8ea1-4dd262109fb2 service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:03:11.432354 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357abe0c-7e50-41bb-8ea1-4dd262109fb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:03:11.432630 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357abe0c-7e50-41bb-8ea1-4dd262109fb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:03:11.434482 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-357abe0c-7e50-41bb-8ea1-4dd262109fb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:03:11.512302 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357abe0c-7e50-41bb-8ea1-4dd262109fb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:03:11.513764 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357abe0c-7e50-41bb-8ea1-4dd262109fb2 service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:03:11.525121 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1268/2532] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:03:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:03:15.366506 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef7f9d6-6ecd-4ae6-a5f1-6ee047e6a61e service cinder] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:03:15.379693 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef7f9d6-6ecd-4ae6-a5f1-6ee047e6a61e service cinder] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:03:15.384768 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b3ca16-d0fa-4179-9e2b-8dc293161a6b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70f704a231a5474e8067a6c2dd88b355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:03:15.385425 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b3ca16-d0fa-4179-9e2b-8dc293161a6b service placement] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:03:15.385708 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b3ca16-d0fa-4179-9e2b-8dc293161a6b service placement] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:03:15.386197 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b3ca16-d0fa-4179-9e2b-8dc293161a6b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:03:15.388256 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-90b3ca16-d0fa-4179-9e2b-8dc293161a6b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:03:15.401761 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b3ca16-d0fa-4179-9e2b-8dc293161a6b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:03:15.403714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b3ca16-d0fa-4179-9e2b-8dc293161a6b service placement] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:03:15.415389 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1265/2533] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:03:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:03:19.518341 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357abe0c-7e50-41bb-8ea1-4dd262109fb2 service neutron] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:03:19.519939 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-357abe0c-7e50-41bb-8ea1-4dd262109fb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:03:19.534133 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357abe0c-7e50-41bb-8ea1-4dd262109fb2 service neutron] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:03:19.538953 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4575bf2f-d5b3-4e24-9832-b176366b1a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:03:19.539889 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4575bf2f-d5b3-4e24-9832-b176366b1a35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:03:19.540176 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4575bf2f-d5b3-4e24-9832-b176366b1a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:03:19.540455 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4575bf2f-d5b3-4e24-9832-b176366b1a35 admin admin] PATH_INFO: `/v3/users/6472100830a54d6a8e21ee55b68e6a02` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:03:19.541666 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4575bf2f-d5b3-4e24-9832-b176366b1a35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6472100830a54d6a8e21ee55b68e6a02)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:03:19.543683 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4575bf2f-d5b3-4e24-9832-b176366b1a35 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:03:19.748975 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-4575bf2f-d5b3-4e24-9832-b176366b1a35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6472100830a54d6a8e21ee55b68e6a02'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:03:19.825669 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-4575bf2f-d5b3-4e24-9832-b176366b1a35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6472100830a54d6a8e21ee55b68e6a02'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:03:19.826321 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-4575bf2f-d5b3-4e24-9832-b176366b1a35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6472100830a54d6a8e21ee55b68e6a02'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:03:19.826969 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-4575bf2f-d5b3-4e24-9832-b176366b1a35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6472100830a54d6a8e21ee55b68e6a02'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:03:19.835414 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-4575bf2f-d5b3-4e24-9832-b176366b1a35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6472100830a54d6a8e21ee55b68e6a02'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:03:19.857550 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1269/2534] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 17:03:19 2022] DELETE /identity/v3/users/6472100830a54d6a8e21ee55b68e6a02 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:03:20.078565 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b3ca16-d0fa-4179-9e2b-8dc293161a6b service placement] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:03:20.091628 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b3ca16-d0fa-4179-9e2b-8dc293161a6b service placement] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:03:20.151012 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf7fe53-87a8-4781-b00f-0d8b8575044e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:03:20.152027 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf7fe53-87a8-4781-b00f-0d8b8575044e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:03:20.152448 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf7fe53-87a8-4781-b00f-0d8b8575044e admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:03:20.152736 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf7fe53-87a8-4781-b00f-0d8b8575044e admin admin] PATH_INFO: `/v3/projects/5d70fe1437fc423681962ee747f07410` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:03:20.154134 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf7fe53-87a8-4781-b00f-0d8b8575044e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d70fe1437fc423681962ee747f07410)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:03:20.156481 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf7fe53-87a8-4781-b00f-0d8b8575044e admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:03:20.283246 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-caf7fe53-87a8-4781-b00f-0d8b8575044e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d70fe1437fc423681962ee747f07410', 'request_id': 'req-caf7fe53-87a8-4781-b00f-0d8b8575044e'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:03:20.288906 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-caf7fe53-87a8-4781-b00f-0d8b8575044e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d70fe1437fc423681962ee747f07410', 'request_id': 'req-caf7fe53-87a8-4781-b00f-0d8b8575044e'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:03:20.295138 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-caf7fe53-87a8-4781-b00f-0d8b8575044e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d70fe1437fc423681962ee747f07410', 'request_id': 'req-caf7fe53-87a8-4781-b00f-0d8b8575044e'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:03:20.454234 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-caf7fe53-87a8-4781-b00f-0d8b8575044e admin admin] The token cache is being invalidate because project 5d70fe1437fc423681962ee747f07410 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 17:03:20.454440 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-caf7fe53-87a8-4781-b00f-0d8b8575044e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:03:20.456038 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-caf7fe53-87a8-4781-b00f-0d8b8575044e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:03:20.462859 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1266/2535] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 17:03:20 2022] DELETE /identity/v3/projects/5d70fe1437fc423681962ee747f07410 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:03:20.463019 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4575bf2f-d5b3-4e24-9832-b176366b1a35 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:03:20.463126 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-4575bf2f-d5b3-4e24-9832-b176366b1a35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:03:20.479731 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4575bf2f-d5b3-4e24-9832-b176366b1a35 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:03:20.539424 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81efef6c-83b6-4064-aa64-d52dfdd0ec8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:03:20.540492 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81efef6c-83b6-4064-aa64-d52dfdd0ec8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:03:20.540777 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81efef6c-83b6-4064-aa64-d52dfdd0ec8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:03:20.541058 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81efef6c-83b6-4064-aa64-d52dfdd0ec8a admin admin] PATH_INFO: `/v3/users/9e337289fc9d40e5877f6f3adc8fd8ad` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:03:20.544278 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:03:20.557283 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81efef6c-83b6-4064-aa64-d52dfdd0ec8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e337289fc9d40e5877f6f3adc8fd8ad)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:03:20.560863 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81efef6c-83b6-4064-aa64-d52dfdd0ec8a admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:03:20.720966 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-81efef6c-83b6-4064-aa64-d52dfdd0ec8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e337289fc9d40e5877f6f3adc8fd8ad'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:03:20.814001 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-81efef6c-83b6-4064-aa64-d52dfdd0ec8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e337289fc9d40e5877f6f3adc8fd8ad'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:03:20.814647 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-81efef6c-83b6-4064-aa64-d52dfdd0ec8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e337289fc9d40e5877f6f3adc8fd8ad'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:03:20.815869 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-81efef6c-83b6-4064-aa64-d52dfdd0ec8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e337289fc9d40e5877f6f3adc8fd8ad'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:03:20.824387 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-81efef6c-83b6-4064-aa64-d52dfdd0ec8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e337289fc9d40e5877f6f3adc8fd8ad'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:03:20.848192 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1270/2536] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 17:03:20 2022] DELETE /identity/v3/users/9e337289fc9d40e5877f6f3adc8fd8ad => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:03:21.106787 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf7fe53-87a8-4781-b00f-0d8b8575044e admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:03:21.108294 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-caf7fe53-87a8-4781-b00f-0d8b8575044e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:03:21.126073 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf7fe53-87a8-4781-b00f-0d8b8575044e admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:03:21.186184 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bdab0f-47a9-4d84-9db3-3c942c105d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:03:21.186939 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bdab0f-47a9-4d84-9db3-3c942c105d06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:03:21.187223 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bdab0f-47a9-4d84-9db3-3c942c105d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:03:21.187515 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bdab0f-47a9-4d84-9db3-3c942c105d06 admin admin] PATH_INFO: `/v3/projects/13e5cd3bf86848afaec07d84e3fc0f69` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:03:21.198814 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bdab0f-47a9-4d84-9db3-3c942c105d06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13e5cd3bf86848afaec07d84e3fc0f69)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:03:21.200791 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bdab0f-47a9-4d84-9db3-3c942c105d06 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:03:21.206832 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:03:21.312690 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-f8bdab0f-47a9-4d84-9db3-3c942c105d06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13e5cd3bf86848afaec07d84e3fc0f69', 'request_id': 'req-f8bdab0f-47a9-4d84-9db3-3c942c105d06'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:03:21.318495 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-f8bdab0f-47a9-4d84-9db3-3c942c105d06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13e5cd3bf86848afaec07d84e3fc0f69', 'request_id': 'req-f8bdab0f-47a9-4d84-9db3-3c942c105d06'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:03:21.325186 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-f8bdab0f-47a9-4d84-9db3-3c942c105d06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13e5cd3bf86848afaec07d84e3fc0f69', 'request_id': 'req-f8bdab0f-47a9-4d84-9db3-3c942c105d06'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:03:21.379055 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-f8bdab0f-47a9-4d84-9db3-3c942c105d06 admin admin] The token cache is being invalidate because project 13e5cd3bf86848afaec07d84e3fc0f69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 17:03:21.379337 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-f8bdab0f-47a9-4d84-9db3-3c942c105d06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:03:21.379988 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-f8bdab0f-47a9-4d84-9db3-3c942c105d06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:03:21.381662 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1267/2537] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 17:03:21 2022] DELETE /identity/v3/projects/13e5cd3bf86848afaec07d84e3fc0f69 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:03:30.843605 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=30,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:03:31.375915 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:04:31.288948 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81efef6c-83b6-4064-aa64-d52dfdd0ec8a admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:04:31.290488 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-81efef6c-83b6-4064-aa64-d52dfdd0ec8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:04:31.414358 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81efef6c-83b6-4064-aa64-d52dfdd0ec8a admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:04:31.473834 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0597f19-e6c4-42fe-b2d6-f9f7f941c340 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:04:31.474621 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0597f19-e6c4-42fe-b2d6-f9f7f941c340 service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:04:31.474913 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0597f19-e6c4-42fe-b2d6-f9f7f941c340 service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:04:31.475186 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0597f19-e6c4-42fe-b2d6-f9f7f941c340 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:04:31.477066 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-b0597f19-e6c4-42fe-b2d6-f9f7f941c340 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:04:31.545892 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0597f19-e6c4-42fe-b2d6-f9f7f941c340 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:04:31.547544 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0597f19-e6c4-42fe-b2d6-f9f7f941c340 service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:04:31.558608 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1271/2538] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:04:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:04:31.912204 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bdab0f-47a9-4d84-9db3-3c942c105d06 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:04:31.913729 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f8bdab0f-47a9-4d84-9db3-3c942c105d06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:04:32.007272 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bdab0f-47a9-4d84-9db3-3c942c105d06 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:04:32.012164 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa9702f-b539-4518-8c5e-ba6ecfbde781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:04:32.013050 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa9702f-b539-4518-8c5e-ba6ecfbde781 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:04:32.013334 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa9702f-b539-4518-8c5e-ba6ecfbde781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:04:32.014103 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa9702f-b539-4518-8c5e-ba6ecfbde781 admin admin] PATH_INFO: `/v3/users/89def089e134402eb097529bcb112ca1` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:04:32.015721 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa9702f-b539-4518-8c5e-ba6ecfbde781 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89def089e134402eb097529bcb112ca1)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:04:32.017705 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa9702f-b539-4518-8c5e-ba6ecfbde781 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:04:32.202475 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-aaa9702f-b539-4518-8c5e-ba6ecfbde781 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89def089e134402eb097529bcb112ca1'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:04:32.277539 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-aaa9702f-b539-4518-8c5e-ba6ecfbde781 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89def089e134402eb097529bcb112ca1'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:04:32.291271 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-aaa9702f-b539-4518-8c5e-ba6ecfbde781 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89def089e134402eb097529bcb112ca1'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:04:32.311181 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-aaa9702f-b539-4518-8c5e-ba6ecfbde781 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89def089e134402eb097529bcb112ca1'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:04:32.313759 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-aaa9702f-b539-4518-8c5e-ba6ecfbde781 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89def089e134402eb097529bcb112ca1'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:04:32.318928 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1268/2539] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 17:04:31 2022] DELETE /identity/v3/users/89def089e134402eb097529bcb112ca1 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:04:32.324146 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc89cf1-ce5b-4f3f-9c5c-c355cb1c69da None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:04:32.324776 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc89cf1-ce5b-4f3f-9c5c-c355cb1c69da None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:04:32.324946 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc89cf1-ce5b-4f3f-9c5c-c355cb1c69da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:04:32.345511 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-6fc89cf1-ce5b-4f3f-9c5c-c355cb1c69da None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:04:32.346761 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-6fc89cf1-ce5b-4f3f-9c5c-c355cb1c69da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:04:32.384677 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:04:32.420742 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1272/2540] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:04:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:04:32.432297 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa9702f-b539-4518-8c5e-ba6ecfbde781 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:04:32.434477 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-aaa9702f-b539-4518-8c5e-ba6ecfbde781 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:04:32.505225 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa9702f-b539-4518-8c5e-ba6ecfbde781 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:04:32.509006 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94edcf56-b321-47bb-8dac-af7bfaed3e3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:04:32.509677 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94edcf56-b321-47bb-8dac-af7bfaed3e3a service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:04:32.509961 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94edcf56-b321-47bb-8dac-af7bfaed3e3a service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:04:32.510239 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94edcf56-b321-47bb-8dac-af7bfaed3e3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:04:32.520447 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94edcf56-b321-47bb-8dac-af7bfaed3e3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:04:32.521948 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94edcf56-b321-47bb-8dac-af7bfaed3e3a service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:04:32.536429 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1269/2541] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:04:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:04:33.045801 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc89cf1-ce5b-4f3f-9c5c-c355cb1c69da None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:04:33.047245 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-6fc89cf1-ce5b-4f3f-9c5c-c355cb1c69da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:04:33.061944 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc89cf1-ce5b-4f3f-9c5c-c355cb1c69da None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:04:33.068801 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea0c242-5c10-4459-b9c9-b59ad21eaaf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:04:33.069491 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea0c242-5c10-4459-b9c9-b59ad21eaaf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:04:33.069865 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea0c242-5c10-4459-b9c9-b59ad21eaaf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:04:33.070112 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea0c242-5c10-4459-b9c9-b59ad21eaaf5 admin admin] PATH_INFO: `/v3/projects/aaeb16bbae374566926cf8512b00fc73` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:04:33.071435 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea0c242-5c10-4459-b9c9-b59ad21eaaf5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aaeb16bbae374566926cf8512b00fc73)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:04:33.074039 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea0c242-5c10-4459-b9c9-b59ad21eaaf5 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:04:33.240007 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-9ea0c242-5c10-4459-b9c9-b59ad21eaaf5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aaeb16bbae374566926cf8512b00fc73', 'request_id': 'req-9ea0c242-5c10-4459-b9c9-b59ad21eaaf5'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:04:33.246984 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-9ea0c242-5c10-4459-b9c9-b59ad21eaaf5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aaeb16bbae374566926cf8512b00fc73', 'request_id': 'req-9ea0c242-5c10-4459-b9c9-b59ad21eaaf5'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:04:33.323442 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-9ea0c242-5c10-4459-b9c9-b59ad21eaaf5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aaeb16bbae374566926cf8512b00fc73', 'request_id': 'req-9ea0c242-5c10-4459-b9c9-b59ad21eaaf5'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:04:33.344377 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-9ea0c242-5c10-4459-b9c9-b59ad21eaaf5 admin admin] The token cache is being invalidate because project aaeb16bbae374566926cf8512b00fc73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 17:04:33.345089 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-9ea0c242-5c10-4459-b9c9-b59ad21eaaf5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:04:33.346101 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-9ea0c242-5c10-4459-b9c9-b59ad21eaaf5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:04:33.349496 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1273/2542] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 17:04:33 2022] DELETE /identity/v3/projects/aaeb16bbae374566926cf8512b00fc73 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:04:42.528134 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:04:43.337279 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=37,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:05:05.759485 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94edcf56-b321-47bb-8dac-af7bfaed3e3a service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:05:05.761017 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-94edcf56-b321-47bb-8dac-af7bfaed3e3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:05:05.856854 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94edcf56-b321-47bb-8dac-af7bfaed3e3a service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:05:05.917122 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ce583f-da3d-40d9-8a81-a8e38d1e5122 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70f704a231a5474e8067a6c2dd88b355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:05:05.917773 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce583f-da3d-40d9-8a81-a8e38d1e5122 service placement] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:05:05.918062 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce583f-da3d-40d9-8a81-a8e38d1e5122 service placement] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:05:05.918340 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce583f-da3d-40d9-8a81-a8e38d1e5122 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:05:05.920127 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-b7ce583f-da3d-40d9-8a81-a8e38d1e5122 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:05:05.932188 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ce583f-da3d-40d9-8a81-a8e38d1e5122 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:05:05.933801 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ce583f-da3d-40d9-8a81-a8e38d1e5122 service placement] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:05:05.945192 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1270/2543] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:05:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:05:15.940564 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=67 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:06:32.190962 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea0c242-5c10-4459-b9c9-b59ad21eaaf5 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:06:32.192366 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-9ea0c242-5c10-4459-b9c9-b59ad21eaaf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:06:32.254235 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea0c242-5c10-4459-b9c9-b59ad21eaaf5 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:06:32.257915 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bac2435-33c4-416a-8ca8-1ae8d1cd4f88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:06:32.258586 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac2435-33c4-416a-8ca8-1ae8d1cd4f88 service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:06:32.258868 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac2435-33c4-416a-8ca8-1ae8d1cd4f88 service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:06:32.259174 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac2435-33c4-416a-8ca8-1ae8d1cd4f88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:06:32.261148 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-7bac2435-33c4-416a-8ca8-1ae8d1cd4f88 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:06:32.340836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bac2435-33c4-416a-8ca8-1ae8d1cd4f88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:06:32.342418 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bac2435-33c4-416a-8ca8-1ae8d1cd4f88 service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:06:32.364205 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1274/2544] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:06:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:06:32.409552 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ce583f-da3d-40d9-8a81-a8e38d1e5122 service placement] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:06:32.420465 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ce583f-da3d-40d9-8a81-a8e38d1e5122 service placement] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:06:32.424059 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfc038e-e27b-4025-b6b8-f20652416d53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:06:32.424695 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc038e-e27b-4025-b6b8-f20652416d53 service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:06:32.424976 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc038e-e27b-4025-b6b8-f20652416d53 service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:06:32.425250 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc038e-e27b-4025-b6b8-f20652416d53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:06:32.427049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-2cfc038e-e27b-4025-b6b8-f20652416d53 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:06:32.502122 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfc038e-e27b-4025-b6b8-f20652416d53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:06:32.503658 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfc038e-e27b-4025-b6b8-f20652416d53 service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:06:32.531395 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1271/2545] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:06:32 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:06:42.361222 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=56 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:06:42.528603 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=36 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:07:07.603387 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bac2435-33c4-416a-8ca8-1ae8d1cd4f88 service cinder] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:07:07.604837 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-7bac2435-33c4-416a-8ca8-1ae8d1cd4f88 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:07:07.683707 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bac2435-33c4-416a-8ca8-1ae8d1cd4f88 service cinder] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:07:07.688011 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cbbd08-e78b-476d-b656-4ed0fc8fdda9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:07:07.688669 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cbbd08-e78b-476d-b656-4ed0fc8fdda9 service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:07:07.688956 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cbbd08-e78b-476d-b656-4ed0fc8fdda9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:07:07.689235 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cbbd08-e78b-476d-b656-4ed0fc8fdda9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:07:07.691059 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-c5cbbd08-e78b-476d-b656-4ed0fc8fdda9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:07:07.701480 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cbbd08-e78b-476d-b656-4ed0fc8fdda9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:07:07.702902 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cbbd08-e78b-476d-b656-4ed0fc8fdda9 service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:07:07.711071 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1275/2546] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:07:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:07:17.707373 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=35 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:07:23.519518 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfc038e-e27b-4025-b6b8-f20652416d53 service cinder] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:07:23.520923 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-2cfc038e-e27b-4025-b6b8-f20652416d53 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:07:23.544277 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfc038e-e27b-4025-b6b8-f20652416d53 service cinder] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:07:23.548935 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd8d86e-341c-4d20-ad70-3bbbd5d49cd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:07:23.549574 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd8d86e-341c-4d20-ad70-3bbbd5d49cd1 service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:07:23.549865 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd8d86e-341c-4d20-ad70-3bbbd5d49cd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:07:23.550141 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd8d86e-341c-4d20-ad70-3bbbd5d49cd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:07:23.551945 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-3dd8d86e-341c-4d20-ad70-3bbbd5d49cd1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:07:23.565498 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application [None req-3dd8d86e-341c-4d20-ad70-3bbbd5d49cd1 service cinder] Could not find project: 5d70fe1437fc423681962ee747f07410.: keystone.exception.ProjectNotFound: Could not find project: 5d70fe1437fc423681962ee747f07410. Jul 18 17:07:23.565498 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 17:07:23.565498 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 17:07:23.565498 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 17:07:23.565498 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 17:07:23.565498 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 17:07:23.565498 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 17:07:23.565498 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 17:07:23.565498 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 17:07:23.565498 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 17:07:23.565498 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 17:07:23.565498 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 17:07:23.565498 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Jul 18 17:07:23.565498 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Jul 18 17:07:23.565498 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Jul 18 17:07:23.566811 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Jul 18 17:07:23.566811 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Jul 18 17:07:23.566811 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Jul 18 17:07:23.566811 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 17:07:23.566811 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 17:07:23.566811 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Jul 18 17:07:23.566811 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Jul 18 17:07:23.566811 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 17:07:23.566811 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 17:07:23.566811 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 17:07:23.566811 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 17:07:23.566811 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 17:07:23.566811 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 17:07:23.566811 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 17:07:23.566811 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 17:07:23.566811 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 17:07:23.566811 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 17:07:23.567990 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 17:07:23.567990 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 17:07:23.567990 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 17:07:23.567990 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 17:07:23.567990 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Jul 18 17:07:23.567990 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Jul 18 17:07:23.567990 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Jul 18 17:07:23.567990 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self._validate_token_resources() Jul 18 17:07:23.567990 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Jul 18 17:07:23.567990 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Jul 18 17:07:23.567990 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Jul 18 17:07:23.567990 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Jul 18 17:07:23.567990 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 17:07:23.567990 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 17:07:23.567990 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 17:07:23.567990 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 17:07:23.567990 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 17:07:23.569183 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 17:07:23.569183 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 17:07:23.569183 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application with Lock( Jul 18 17:07:23.569183 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 17:07:23.569183 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._enter() Jul 18 17:07:23.569183 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 17:07:23.569183 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 17:07:23.569183 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 17:07:23.569183 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.creator() Jul 18 17:07:23.569183 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 17:07:23.569183 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application created_value = creator( Jul 18 17:07:23.569183 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 17:07:23.569183 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 17:07:23.569183 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 17:07:23.569183 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 17:07:23.569183 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 17:07:23.569183 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 17:07:23.570205 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 5d70fe1437fc423681962ee747f07410. Jul 18 17:07:23.570205 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: ERROR keystone.server.flask.application Jul 18 17:07:23.570205 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1272/2547] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:07:23 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 48 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Jul 18 17:07:23.636684 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cbbd08-e78b-476d-b656-4ed0fc8fdda9 service neutron] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:07:23.638152 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-c5cbbd08-e78b-476d-b656-4ed0fc8fdda9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:07:23.743125 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cbbd08-e78b-476d-b656-4ed0fc8fdda9 service neutron] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:07:23.747889 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413a1f75-61e1-48a7-90c0-f453e220e35c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:07:23.748762 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a1f75-61e1-48a7-90c0-f453e220e35c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:07:23.749041 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a1f75-61e1-48a7-90c0-f453e220e35c admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:07:23.749375 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a1f75-61e1-48a7-90c0-f453e220e35c admin admin] PATH_INFO: `/v3/users/c7c3d91cd21744548a085f2339a08812` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:07:23.764507 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413a1f75-61e1-48a7-90c0-f453e220e35c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7c3d91cd21744548a085f2339a08812)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:07:23.767129 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413a1f75-61e1-48a7-90c0-f453e220e35c admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:07:23.990369 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-413a1f75-61e1-48a7-90c0-f453e220e35c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7c3d91cd21744548a085f2339a08812'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:07:24.073006 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-413a1f75-61e1-48a7-90c0-f453e220e35c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7c3d91cd21744548a085f2339a08812'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:07:24.073641 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-413a1f75-61e1-48a7-90c0-f453e220e35c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7c3d91cd21744548a085f2339a08812'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:07:24.074255 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-413a1f75-61e1-48a7-90c0-f453e220e35c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7c3d91cd21744548a085f2339a08812'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:07:24.083318 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-413a1f75-61e1-48a7-90c0-f453e220e35c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7c3d91cd21744548a085f2339a08812'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:07:24.103074 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1276/2548] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 17:07:23 2022] DELETE /identity/v3/users/c7c3d91cd21744548a085f2339a08812 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:07:24.107647 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df87583b-50e0-4476-8d64-c24083868080 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:07:24.107941 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df87583b-50e0-4476-8d64-c24083868080 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:07:24.108224 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df87583b-50e0-4476-8d64-c24083868080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:07:24.138490 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-df87583b-50e0-4476-8d64-c24083868080 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:07:24.139722 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-df87583b-50e0-4476-8d64-c24083868080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:07:24.199672 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1273/2549] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:07:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:07:24.210542 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413a1f75-61e1-48a7-90c0-f453e220e35c admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:07:24.212029 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-413a1f75-61e1-48a7-90c0-f453e220e35c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:07:24.257387 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:07:24.272206 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413a1f75-61e1-48a7-90c0-f453e220e35c admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:07:24.276218 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cd75f8-4fce-4b10-8369-a70a1bbf113f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:07:24.280696 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd75f8-4fce-4b10-8369-a70a1bbf113f service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:07:24.280696 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd75f8-4fce-4b10-8369-a70a1bbf113f service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:07:24.280696 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd75f8-4fce-4b10-8369-a70a1bbf113f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:07:24.287140 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cd75f8-4fce-4b10-8369-a70a1bbf113f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:07:24.288691 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cd75f8-4fce-4b10-8369-a70a1bbf113f service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:07:24.298928 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1277/2550] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:07:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:07:24.725959 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df87583b-50e0-4476-8d64-c24083868080 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:07:24.732224 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-df87583b-50e0-4476-8d64-c24083868080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:07:24.741593 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df87583b-50e0-4476-8d64-c24083868080 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:07:24.747919 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524b8a68-ae9b-4026-9f98-9940ebb46d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:07:24.749390 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b8a68-ae9b-4026-9f98-9940ebb46d63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:07:24.749886 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b8a68-ae9b-4026-9f98-9940ebb46d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:07:24.750384 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b8a68-ae9b-4026-9f98-9940ebb46d63 admin admin] PATH_INFO: `/v3/projects/0396828d8a6c423c85c3105cf2776760` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:07:24.765304 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524b8a68-ae9b-4026-9f98-9940ebb46d63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0396828d8a6c423c85c3105cf2776760)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:07:24.767430 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524b8a68-ae9b-4026-9f98-9940ebb46d63 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:07:24.889536 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-524b8a68-ae9b-4026-9f98-9940ebb46d63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0396828d8a6c423c85c3105cf2776760', 'request_id': 'req-524b8a68-ae9b-4026-9f98-9940ebb46d63'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:07:24.895555 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-524b8a68-ae9b-4026-9f98-9940ebb46d63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0396828d8a6c423c85c3105cf2776760', 'request_id': 'req-524b8a68-ae9b-4026-9f98-9940ebb46d63'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:07:24.904164 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-524b8a68-ae9b-4026-9f98-9940ebb46d63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0396828d8a6c423c85c3105cf2776760', 'request_id': 'req-524b8a68-ae9b-4026-9f98-9940ebb46d63'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:07:25.004627 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-524b8a68-ae9b-4026-9f98-9940ebb46d63 admin admin] The token cache is being invalidate because project 0396828d8a6c423c85c3105cf2776760 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 17:07:25.007913 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-524b8a68-ae9b-4026-9f98-9940ebb46d63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:07:25.007913 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-524b8a68-ae9b-4026-9f98-9940ebb46d63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:07:25.007913 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1274/2551] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 17:07:24 2022] DELETE /identity/v3/projects/0396828d8a6c423c85c3105cf2776760 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:07:34.512410 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=8 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:07:35.000669 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=70,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:08:31.419003 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cd75f8-4fce-4b10-8369-a70a1bbf113f service neutron] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:08:31.420615 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-86cd75f8-4fce-4b10-8369-a70a1bbf113f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:08:31.432257 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cd75f8-4fce-4b10-8369-a70a1bbf113f service neutron] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:08:31.482047 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef288cb-010b-4895-8576-8c21971c15b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:08:31.482707 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef288cb-010b-4895-8576-8c21971c15b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:08:31.482991 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef288cb-010b-4895-8576-8c21971c15b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:08:31.483269 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef288cb-010b-4895-8576-8c21971c15b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:08:31.485108 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-9ef288cb-010b-4895-8576-8c21971c15b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:08:31.576340 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef288cb-010b-4895-8576-8c21971c15b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:08:31.577849 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef288cb-010b-4895-8576-8c21971c15b5 service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:08:31.599378 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1278/2552] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:08:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:08:32.772621 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524b8a68-ae9b-4026-9f98-9940ebb46d63 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:08:32.774046 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-524b8a68-ae9b-4026-9f98-9940ebb46d63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:08:32.787806 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524b8a68-ae9b-4026-9f98-9940ebb46d63 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:08:32.791386 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf719e9-22f0-4cbf-a16c-8722018e1be5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:08:32.792057 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf719e9-22f0-4cbf-a16c-8722018e1be5 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:08:32.792338 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf719e9-22f0-4cbf-a16c-8722018e1be5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:08:32.792615 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf719e9-22f0-4cbf-a16c-8722018e1be5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:08:32.794534 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-3bf719e9-22f0-4cbf-a16c-8722018e1be5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:08:32.813927 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf719e9-22f0-4cbf-a16c-8722018e1be5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:08:32.815392 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf719e9-22f0-4cbf-a16c-8722018e1be5 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:08:32.828338 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1275/2553] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:08:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:08:41.596526 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=66 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:08:42.822747 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=6 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:09:25.903308 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef288cb-010b-4895-8576-8c21971c15b5 service neutron] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:09:25.905319 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-9ef288cb-010b-4895-8576-8c21971c15b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:09:25.920678 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef288cb-010b-4895-8576-8c21971c15b5 service neutron] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:09:25.927330 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c29142-8cfe-48bc-b853-31fa5293bd02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:09:25.928371 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c29142-8cfe-48bc-b853-31fa5293bd02 service nova] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:09:25.929137 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c29142-8cfe-48bc-b853-31fa5293bd02 service nova] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:09:25.929137 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c29142-8cfe-48bc-b853-31fa5293bd02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:09:25.931383 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-a3c29142-8cfe-48bc-b853-31fa5293bd02 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:09:25.949161 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c29142-8cfe-48bc-b853-31fa5293bd02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:09:25.951367 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c29142-8cfe-48bc-b853-31fa5293bd02 service nova] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:09:25.964398 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1279/2554] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:09:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:09:31.379096 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf719e9-22f0-4cbf-a16c-8722018e1be5 service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:09:31.380533 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-3bf719e9-22f0-4cbf-a16c-8722018e1be5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:09:31.395024 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf719e9-22f0-4cbf-a16c-8722018e1be5 service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:09:31.400652 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a68feb-be73-4255-af4d-65412946c539 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:09:31.401235 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a68feb-be73-4255-af4d-65412946c539 service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:09:31.401518 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a68feb-be73-4255-af4d-65412946c539 service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:09:31.401794 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a68feb-be73-4255-af4d-65412946c539 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:09:31.404385 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e0a68feb-be73-4255-af4d-65412946c539 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:09:31.484930 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a68feb-be73-4255-af4d-65412946c539 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:09:31.486568 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a68feb-be73-4255-af4d-65412946c539 service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:09:31.505955 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1276/2555] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:09:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:09:35.958539 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=6 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:09:41.503220 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=41 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:10:10.825941 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c29142-8cfe-48bc-b853-31fa5293bd02 service nova] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:10:10.827462 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-a3c29142-8cfe-48bc-b853-31fa5293bd02 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:10:10.906065 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c29142-8cfe-48bc-b853-31fa5293bd02 service nova] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:10:10.911155 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680d84ab-da3b-4314-a3de-9deccf578d91 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70f704a231a5474e8067a6c2dd88b355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:10:10.911902 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d84ab-da3b-4314-a3de-9deccf578d91 service placement] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:10:10.912193 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d84ab-da3b-4314-a3de-9deccf578d91 service placement] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:10:10.912471 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d84ab-da3b-4314-a3de-9deccf578d91 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:10:10.914256 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-680d84ab-da3b-4314-a3de-9deccf578d91 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:10:10.927398 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680d84ab-da3b-4314-a3de-9deccf578d91 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:10:10.929061 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680d84ab-da3b-4314-a3de-9deccf578d91 service placement] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:10:10.939064 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1280/2556] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:10:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 17:10:20.934711 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=32 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:11:33.411645 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a68feb-be73-4255-af4d-65412946c539 service nova] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:11:33.423436 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a68feb-be73-4255-af4d-65412946c539 service nova] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:11:33.429565 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a0ca11-1b47-4f95-b00b-a3728c937d5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:11:33.430854 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a0ca11-1b47-4f95-b00b-a3728c937d5d service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:11:33.431147 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a0ca11-1b47-4f95-b00b-a3728c937d5d service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:11:33.431428 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a0ca11-1b47-4f95-b00b-a3728c937d5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:11:33.433347 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-58a0ca11-1b47-4f95-b00b-a3728c937d5d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:11:33.508261 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a0ca11-1b47-4f95-b00b-a3728c937d5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:11:33.511184 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a0ca11-1b47-4f95-b00b-a3728c937d5d service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:11:33.522382 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1277/2557] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:11:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:11:43.517806 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=33 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:12:09.570631 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680d84ab-da3b-4314-a3de-9deccf578d91 service placement] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:12:09.572266 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-680d84ab-da3b-4314-a3de-9deccf578d91 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:12:09.584541 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680d84ab-da3b-4314-a3de-9deccf578d91 service placement] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:12:09.592682 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f6bd63-23c5-4033-bba9-922656856586 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:12:09.592682 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f6bd63-23c5-4033-bba9-922656856586 service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:12:09.592682 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f6bd63-23c5-4033-bba9-922656856586 service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:12:09.592682 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f6bd63-23c5-4033-bba9-922656856586 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:12:09.593076 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-01f6bd63-23c5-4033-bba9-922656856586 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:12:09.601752 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f6bd63-23c5-4033-bba9-922656856586 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:12:09.603175 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f6bd63-23c5-4033-bba9-922656856586 service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:12:09.611617 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1281/2558] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:12:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 17:12:19.607942 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=6 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:12:32.928994 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a0ca11-1b47-4f95-b00b-a3728c937d5d service nova] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:12:32.931229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-58a0ca11-1b47-4f95-b00b-a3728c937d5d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:12:32.990175 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a0ca11-1b47-4f95-b00b-a3728c937d5d service nova] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:12:32.993892 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d36513b-1a13-4d79-9577-b4c98d0da1d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:12:32.994534 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d36513b-1a13-4d79-9577-b4c98d0da1d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:12:32.994817 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d36513b-1a13-4d79-9577-b4c98d0da1d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:12:32.995094 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d36513b-1a13-4d79-9577-b4c98d0da1d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:12:32.998426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-8d36513b-1a13-4d79-9577-b4c98d0da1d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:12:33.088856 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d36513b-1a13-4d79-9577-b4c98d0da1d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:12:33.090960 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d36513b-1a13-4d79-9577-b4c98d0da1d0 service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:12:33.100245 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1278/2559] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:12:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:12:43.096003 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=59 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:13:31.166274 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f6bd63-23c5-4033-bba9-922656856586 service neutron] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:13:31.179136 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f6bd63-23c5-4033-bba9-922656856586 service neutron] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:13:31.183063 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea9de9e-c3fa-46a2-9857-0ec5346567b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:13:31.183716 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9de9e-c3fa-46a2-9857-0ec5346567b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:13:31.183998 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9de9e-c3fa-46a2-9857-0ec5346567b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:13:31.184277 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9de9e-c3fa-46a2-9857-0ec5346567b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:13:31.202186 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea9de9e-c3fa-46a2-9857-0ec5346567b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:13:31.204007 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea9de9e-c3fa-46a2-9857-0ec5346567b8 service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:13:31.215747 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1282/2560] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:13:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:13:41.210053 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=11 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:14:31.049125 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d36513b-1a13-4d79-9577-b4c98d0da1d0 service cinder] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:14:31.065013 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d36513b-1a13-4d79-9577-b4c98d0da1d0 service cinder] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:14:31.070769 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2a2d50-7424-4069-9955-bc8a84eea734 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:14:31.071441 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a2d50-7424-4069-9955-bc8a84eea734 service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:14:31.071749 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a2d50-7424-4069-9955-bc8a84eea734 service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:14:31.072024 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a2d50-7424-4069-9955-bc8a84eea734 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:14:31.081067 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2a2d50-7424-4069-9955-bc8a84eea734 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:14:31.082715 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2a2d50-7424-4069-9955-bc8a84eea734 service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:14:31.092381 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1279/2561] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:14:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:14:35.852073 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea9de9e-c3fa-46a2-9857-0ec5346567b8 service neutron] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:14:35.872655 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea9de9e-c3fa-46a2-9857-0ec5346567b8 service neutron] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:14:35.878693 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415ed256-45d2-43a9-8190-d3db696935dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:14:35.879417 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ed256-45d2-43a9-8190-d3db696935dc service nova] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:14:35.883096 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ed256-45d2-43a9-8190-d3db696935dc service nova] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:14:35.883096 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ed256-45d2-43a9-8190-d3db696935dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:14:35.890407 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415ed256-45d2-43a9-8190-d3db696935dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:14:35.892098 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415ed256-45d2-43a9-8190-d3db696935dc service nova] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:14:35.905454 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1283/2562] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:14:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:14:39.863097 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2a2d50-7424-4069-9955-bc8a84eea734 service nova] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:14:39.880009 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2a2d50-7424-4069-9955-bc8a84eea734 service nova] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:14:39.883890 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ae4f5-bc96-4138-8e71-5a9df3084a8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:14:39.884538 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ae4f5-bc96-4138-8e71-5a9df3084a8a service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:14:39.884827 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ae4f5-bc96-4138-8e71-5a9df3084a8a service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:14:39.885107 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ae4f5-bc96-4138-8e71-5a9df3084a8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:14:39.887564 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-797ae4f5-bc96-4138-8e71-5a9df3084a8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:14:39.973103 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797ae4f5-bc96-4138-8e71-5a9df3084a8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:14:39.973103 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797ae4f5-bc96-4138-8e71-5a9df3084a8a service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:14:39.980426 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1280/2563] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:14:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:14:43.624645 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415ed256-45d2-43a9-8190-d3db696935dc service nova] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:14:43.630552 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-415ed256-45d2-43a9-8190-d3db696935dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:14:43.652761 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415ed256-45d2-43a9-8190-d3db696935dc service nova] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:14:43.658273 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41713644-9d39-4b9c-b4a7-7ebd7d963ace service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70f704a231a5474e8067a6c2dd88b355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:14:43.658969 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41713644-9d39-4b9c-b4a7-7ebd7d963ace service placement] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:14:43.659216 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41713644-9d39-4b9c-b4a7-7ebd7d963ace service placement] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:14:43.659513 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41713644-9d39-4b9c-b4a7-7ebd7d963ace service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:14:43.661453 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-41713644-9d39-4b9c-b4a7-7ebd7d963ace service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:14:43.675960 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41713644-9d39-4b9c-b4a7-7ebd7d963ace service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:14:43.677565 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41713644-9d39-4b9c-b4a7-7ebd7d963ace service placement] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:14:43.687962 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1284/2564] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:14:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:14:43.765925 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ae4f5-bc96-4138-8e71-5a9df3084a8a service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:14:43.767414 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-797ae4f5-bc96-4138-8e71-5a9df3084a8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:14:43.780766 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ae4f5-bc96-4138-8e71-5a9df3084a8a service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:14:43.785409 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e4619b-99d8-4ae7-a405-ec40a5e440e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:14:43.786305 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e4619b-99d8-4ae7-a405-ec40a5e440e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:14:43.786586 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e4619b-99d8-4ae7-a405-ec40a5e440e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:14:43.786865 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e4619b-99d8-4ae7-a405-ec40a5e440e0 admin admin] PATH_INFO: `/v3/users/642ba7eed36e4956834c06b771259a42` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:14:43.788135 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e4619b-99d8-4ae7-a405-ec40a5e440e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=642ba7eed36e4956834c06b771259a42)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:14:43.790101 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e4619b-99d8-4ae7-a405-ec40a5e440e0 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:14:43.974167 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-40e4619b-99d8-4ae7-a405-ec40a5e440e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '642ba7eed36e4956834c06b771259a42'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:14:44.049370 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-40e4619b-99d8-4ae7-a405-ec40a5e440e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '642ba7eed36e4956834c06b771259a42'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:14:44.057002 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-40e4619b-99d8-4ae7-a405-ec40a5e440e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '642ba7eed36e4956834c06b771259a42'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:14:44.069036 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-40e4619b-99d8-4ae7-a405-ec40a5e440e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '642ba7eed36e4956834c06b771259a42'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:14:44.075742 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-40e4619b-99d8-4ae7-a405-ec40a5e440e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '642ba7eed36e4956834c06b771259a42'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:14:44.076559 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1281/2565] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 17:14:43 2022] DELETE /identity/v3/users/642ba7eed36e4956834c06b771259a42 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:14:44.390893 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41713644-9d39-4b9c-b4a7-7ebd7d963ace service placement] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:14:44.403053 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41713644-9d39-4b9c-b4a7-7ebd7d963ace service placement] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:14:44.460429 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca0101f-2bcd-4344-b933-e2430027efba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:14:44.461293 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca0101f-2bcd-4344-b933-e2430027efba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:14:44.461580 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca0101f-2bcd-4344-b933-e2430027efba admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:14:44.461858 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca0101f-2bcd-4344-b933-e2430027efba admin admin] PATH_INFO: `/v3/projects/69c48eddc2334918970e61725bedafd1` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:14:44.463046 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca0101f-2bcd-4344-b933-e2430027efba admin admin] RBAC: Authorizing `identity:delete_project(project_id=69c48eddc2334918970e61725bedafd1)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:14:44.464979 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca0101f-2bcd-4344-b933-e2430027efba admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:14:44.566905 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-3ca0101f-2bcd-4344-b933-e2430027efba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69c48eddc2334918970e61725bedafd1', 'request_id': 'req-3ca0101f-2bcd-4344-b933-e2430027efba'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:14:44.573158 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-3ca0101f-2bcd-4344-b933-e2430027efba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69c48eddc2334918970e61725bedafd1', 'request_id': 'req-3ca0101f-2bcd-4344-b933-e2430027efba'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:14:44.651141 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-3ca0101f-2bcd-4344-b933-e2430027efba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69c48eddc2334918970e61725bedafd1', 'request_id': 'req-3ca0101f-2bcd-4344-b933-e2430027efba'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:14:44.662091 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-3ca0101f-2bcd-4344-b933-e2430027efba admin admin] The token cache is being invalidate because project 69c48eddc2334918970e61725bedafd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 17:14:44.662374 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-3ca0101f-2bcd-4344-b933-e2430027efba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:14:44.662998 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-3ca0101f-2bcd-4344-b933-e2430027efba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:14:44.664594 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1285/2566] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 17:14:44 2022] DELETE /identity/v3/projects/69c48eddc2334918970e61725bedafd1 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:14:44.671097 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cbffb5-1740-4b2c-bfed-f5379fa2726f None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:14:44.671384 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cbffb5-1740-4b2c-bfed-f5379fa2726f None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:14:44.677606 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cbffb5-1740-4b2c-bfed-f5379fa2726f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:14:44.718772 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-69cbffb5-1740-4b2c-bfed-f5379fa2726f None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:14:44.720007 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-69cbffb5-1740-4b2c-bfed-f5379fa2726f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:14:44.724228 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:14:44.780678 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1282/2567] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:14:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:14:44.789422 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5d878-5e5a-435c-91a3-f886cab4b2af None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:14:44.789711 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5d878-5e5a-435c-91a3-f886cab4b2af None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:14:44.789990 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5d878-5e5a-435c-91a3-f886cab4b2af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:14:44.808928 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-eea5d878-5e5a-435c-91a3-f886cab4b2af None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:14:44.810772 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-eea5d878-5e5a-435c-91a3-f886cab4b2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:14:44.818991 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1286/2568] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:14:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:14:44.823303 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cbffb5-1740-4b2c-bfed-f5379fa2726f None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:14:44.833527 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cbffb5-1740-4b2c-bfed-f5379fa2726f None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:14:44.838312 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8aa56b-9924-4e37-a64a-75fe0cd6075a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:14:44.852095 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8aa56b-9924-4e37-a64a-75fe0cd6075a admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:14:44.852095 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8aa56b-9924-4e37-a64a-75fe0cd6075a admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:14:44.852095 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8aa56b-9924-4e37-a64a-75fe0cd6075a admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:14:44.852095 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8aa56b-9924-4e37-a64a-75fe0cd6075a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 17:14:44.852095 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8aa56b-9924-4e37-a64a-75fe0cd6075a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:14:44.852095 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8aa56b-9924-4e37-a64a-75fe0cd6075a admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:14:44.854702 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1283/2569] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 17:14:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 17:14:44.860010 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea5d878-5e5a-435c-91a3-f886cab4b2af None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:14:44.869380 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea5d878-5e5a-435c-91a3-f886cab4b2af None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:14:44.875073 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c9e073-0934-4a16-b7bf-a919838b67bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:14:44.875852 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9e073-0934-4a16-b7bf-a919838b67bc admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:14:44.876138 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9e073-0934-4a16-b7bf-a919838b67bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:14:44.876416 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9e073-0934-4a16-b7bf-a919838b67bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:14:44.877261 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c9e073-0934-4a16-b7bf-a919838b67bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:14:44.879263 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c9e073-0934-4a16-b7bf-a919838b67bc admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:14:45.002098 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1287/2570] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 17:14:44 2022] POST /identity/v3/projects => generated 387 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:14:45.006340 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8aa56b-9924-4e37-a64a-75fe0cd6075a admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:14:45.017050 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8aa56b-9924-4e37-a64a-75fe0cd6075a admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:14:45.075566 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464ab10f-fa3f-4869-9856-4f7a0a020923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:14:45.076314 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464ab10f-fa3f-4869-9856-4f7a0a020923 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:14:45.076606 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464ab10f-fa3f-4869-9856-4f7a0a020923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:14:45.076890 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464ab10f-fa3f-4869-9856-4f7a0a020923 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:14:45.077579 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464ab10f-fa3f-4869-9856-4f7a0a020923 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:14:45.079527 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464ab10f-fa3f-4869-9856-4f7a0a020923 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:14:45.158739 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1284/2571] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 17:14:45 2022] POST /identity/v3/users => generated 294 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:14:45.163019 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c9e073-0934-4a16-b7bf-a919838b67bc admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:14:45.174222 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c9e073-0934-4a16-b7bf-a919838b67bc admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:14:45.179069 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6215663-b309-483a-a650-c6488ffc8db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:14:45.179771 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6215663-b309-483a-a650-c6488ffc8db4 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:14:45.180056 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6215663-b309-483a-a650-c6488ffc8db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:14:45.180328 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6215663-b309-483a-a650-c6488ffc8db4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:14:45.181054 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6215663-b309-483a-a650-c6488ffc8db4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:14:45.182650 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6215663-b309-483a-a650-c6488ffc8db4 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:14:45.188917 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:14:45.194102 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1288/2572] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 17:14:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:14:45.198267 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464ab10f-fa3f-4869-9856-4f7a0a020923 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:14:45.209364 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464ab10f-fa3f-4869-9856-4f7a0a020923 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:14:45.214088 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836f2988-1680-4d54-b3b2-264c325bc479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:14:45.214925 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f2988-1680-4d54-b3b2-264c325bc479 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:14:45.215088 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f2988-1680-4d54-b3b2-264c325bc479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:14:45.215335 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f2988-1680-4d54-b3b2-264c325bc479 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:14:45.216158 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836f2988-1680-4d54-b3b2-264c325bc479 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:14:45.217759 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836f2988-1680-4d54-b3b2-264c325bc479 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:14:45.231451 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1285/2573] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 17:14:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:14:45.232588 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6215663-b309-483a-a650-c6488ffc8db4 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:14:45.243017 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6215663-b309-483a-a650-c6488ffc8db4 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:14:45.248350 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423d49a-c06a-4033-8dd7-d94610836b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:14:45.249248 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423d49a-c06a-4033-8dd7-d94610836b3a admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:14:45.249539 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423d49a-c06a-4033-8dd7-d94610836b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:14:45.249829 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423d49a-c06a-4033-8dd7-d94610836b3a admin admin] PATH_INFO: `/v3/projects/1bad1610198d4b8aacfafc8c92dc7e6d/users/9aec7436e3a549018fd7892969870019/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:14:45.267220 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b423d49a-c06a-4033-8dd7-d94610836b3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bad1610198d4b8aacfafc8c92dc7e6d, user_id=9aec7436e3a549018fd7892969870019, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:14:45.269988 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b423d49a-c06a-4033-8dd7-d94610836b3a admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:14:45.388825 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1289/2574] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 17:14:45 2022] PUT /identity/v3/projects/1bad1610198d4b8aacfafc8c92dc7e6d/users/9aec7436e3a549018fd7892969870019/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 17:14:45.404160 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720c4ef4-8fdf-42a1-94a5-f7818791b5aa None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:14:45.404456 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720c4ef4-8fdf-42a1-94a5-f7818791b5aa None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:14:45.404740 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720c4ef4-8fdf-42a1-94a5-f7818791b5aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:14:45.423708 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-720c4ef4-8fdf-42a1-94a5-f7818791b5aa None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:14:45.424976 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-720c4ef4-8fdf-42a1-94a5-f7818791b5aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:14:45.485451 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1286/2575] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:14:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:14:45.494952 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423d49a-c06a-4033-8dd7-d94610836b3a admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:14:45.497270 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-b423d49a-c06a-4033-8dd7-d94610836b3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:14:45.555050 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423d49a-c06a-4033-8dd7-d94610836b3a admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:14:45.558779 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ce308-d9e5-48be-8646-6872b865e6fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:14:45.560183 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ce308-d9e5-48be-8646-6872b865e6fa service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:14:45.560466 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ce308-d9e5-48be-8646-6872b865e6fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:14:45.560758 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ce308-d9e5-48be-8646-6872b865e6fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:14:45.569683 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59ce308-d9e5-48be-8646-6872b865e6fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:14:45.571106 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59ce308-d9e5-48be-8646-6872b865e6fa service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:14:45.580448 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1290/2576] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:14:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:14:51.019336 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4fc1a7-8bf5-44cb-8bea-09d89fc0ba88 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:14:51.019932 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4fc1a7-8bf5-44cb-8bea-09d89fc0ba88 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:14:51.019932 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4fc1a7-8bf5-44cb-8bea-09d89fc0ba88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:14:51.050332 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-ba4fc1a7-8bf5-44cb-8bea-09d89fc0ba88 None None] MFA Rules not processed for user `9aec7436e3a549018fd7892969870019`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:14:51.051585 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-ba4fc1a7-8bf5-44cb-8bea-09d89fc0ba88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:14:51.058419 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:14:51.124481 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1287/2577] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:14:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:14:51.138399 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ce308-d9e5-48be-8646-6872b865e6fa service neutron] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:14:51.139949 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-b59ce308-d9e5-48be-8646-6872b865e6fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:14:51.221456 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ce308-d9e5-48be-8646-6872b865e6fa service neutron] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:14:51.227146 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c16966-b10d-466f-a9e5-7574e74fa470 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:14:51.227950 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c16966-b10d-466f-a9e5-7574e74fa470 service nova] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:14:51.228233 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c16966-b10d-466f-a9e5-7574e74fa470 service nova] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:14:51.228508 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c16966-b10d-466f-a9e5-7574e74fa470 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:14:51.238065 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c16966-b10d-466f-a9e5-7574e74fa470 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:14:51.239721 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c16966-b10d-466f-a9e5-7574e74fa470 service nova] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:14:51.253321 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1291/2578] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:14:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:14:53.663096 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e7af8-c955-40ac-9e2d-2107b015e9dd None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:14:53.663652 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e7af8-c955-40ac-9e2d-2107b015e9dd None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:14:53.663976 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e7af8-c955-40ac-9e2d-2107b015e9dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:14:53.687381 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-8d6e7af8-c955-40ac-9e2d-2107b015e9dd None None] MFA Rules not processed for user `9aec7436e3a549018fd7892969870019`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:14:53.688657 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-8d6e7af8-c955-40ac-9e2d-2107b015e9dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:14:53.696393 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1288/2579] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:14:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:14:54.032590 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c16966-b10d-466f-a9e5-7574e74fa470 service nova] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:14:54.042282 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c16966-b10d-466f-a9e5-7574e74fa470 service nova] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:14:54.046496 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777cdc8e-5b0d-4438-82f4-8fbad35e6689 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:14:54.047395 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777cdc8e-5b0d-4438-82f4-8fbad35e6689 service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:14:54.047704 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777cdc8e-5b0d-4438-82f4-8fbad35e6689 service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:14:54.047978 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777cdc8e-5b0d-4438-82f4-8fbad35e6689 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:14:54.049928 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-777cdc8e-5b0d-4438-82f4-8fbad35e6689 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:14:54.065500 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777cdc8e-5b0d-4438-82f4-8fbad35e6689 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:14:54.068016 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777cdc8e-5b0d-4438-82f4-8fbad35e6689 service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:14:54.078368 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1292/2580] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:14:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:14:54.567435 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e7af8-c955-40ac-9e2d-2107b015e9dd None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:14:54.569133 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-8d6e7af8-c955-40ac-9e2d-2107b015e9dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:14:54.688977 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e7af8-c955-40ac-9e2d-2107b015e9dd None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:14:54.694765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a829309f-4170-4716-965d-c419977b1754 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70f704a231a5474e8067a6c2dd88b355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:14:54.695442 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a829309f-4170-4716-965d-c419977b1754 service placement] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:14:54.695765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a829309f-4170-4716-965d-c419977b1754 service placement] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:14:54.696050 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a829309f-4170-4716-965d-c419977b1754 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:14:54.697957 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a829309f-4170-4716-965d-c419977b1754 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:14:54.711440 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a829309f-4170-4716-965d-c419977b1754 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:14:54.713198 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a829309f-4170-4716-965d-c419977b1754 service placement] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:14:54.724694 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1289/2581] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:14:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:15:04.073248 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=76,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:15:04.719860 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=60 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:15:04.734347 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777cdc8e-5b0d-4438-82f4-8fbad35e6689 service neutron] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:15:04.748932 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777cdc8e-5b0d-4438-82f4-8fbad35e6689 service neutron] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:15:04.754597 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb6b66-5ee6-40c2-98d8-0988e01d2dd9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:15:04.755250 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb6b66-5ee6-40c2-98d8-0988e01d2dd9 service nova] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:15:04.757197 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb6b66-5ee6-40c2-98d8-0988e01d2dd9 service nova] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:15:04.757197 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb6b66-5ee6-40c2-98d8-0988e01d2dd9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:15:04.757591 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-f3fb6b66-5ee6-40c2-98d8-0988e01d2dd9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:15:04.770899 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fb6b66-5ee6-40c2-98d8-0988e01d2dd9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:15:04.772516 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fb6b66-5ee6-40c2-98d8-0988e01d2dd9 service nova] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:15:04.783690 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1293/2582] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:15:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:15:14.779033 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=6 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:15:16.906188 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a829309f-4170-4716-965d-c419977b1754 service placement] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:15:16.927415 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a829309f-4170-4716-965d-c419977b1754 service placement] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:15:16.932721 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96eca38-be72-4a32-8f0f-577bbfda5580 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70f704a231a5474e8067a6c2dd88b355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:15:16.933375 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96eca38-be72-4a32-8f0f-577bbfda5580 service placement] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:15:16.933664 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96eca38-be72-4a32-8f0f-577bbfda5580 service placement] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:15:16.933938 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96eca38-be72-4a32-8f0f-577bbfda5580 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:15:16.935778 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a96eca38-be72-4a32-8f0f-577bbfda5580 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:15:16.961966 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96eca38-be72-4a32-8f0f-577bbfda5580 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:15:16.963587 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96eca38-be72-4a32-8f0f-577bbfda5580 service placement] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:15:16.975478 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1290/2583] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:15:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:15:26.970314 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=15 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:15:28.418361 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5128b3-81b4-4dc1-9296-8d1466df00d3 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:15:28.418905 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5128b3-81b4-4dc1-9296-8d1466df00d3 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:15:28.418905 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5128b3-81b4-4dc1-9296-8d1466df00d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:15:28.447309 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-ff5128b3-81b4-4dc1-9296-8d1466df00d3 None None] MFA Rules not processed for user `c243a50903b244519ccd58f02cb798da`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:15:28.448629 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-ff5128b3-81b4-4dc1-9296-8d1466df00d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:15:28.504860 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1294/2584] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:15:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:15:38.496799 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=28 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:16:35.443876 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96eca38-be72-4a32-8f0f-577bbfda5580 service placement] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:16:35.445507 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a96eca38-be72-4a32-8f0f-577bbfda5580 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:16:35.527888 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96eca38-be72-4a32-8f0f-577bbfda5580 service placement] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:16:35.531701 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bc4aa9-e681-48b2-b8b1-5e7bfc790d62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:16:35.538812 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc4aa9-e681-48b2-b8b1-5e7bfc790d62 service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:16:35.538812 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc4aa9-e681-48b2-b8b1-5e7bfc790d62 service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:16:35.538812 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc4aa9-e681-48b2-b8b1-5e7bfc790d62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:16:35.538812 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f6bc4aa9-e681-48b2-b8b1-5e7bfc790d62 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:16:35.557198 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bc4aa9-e681-48b2-b8b1-5e7bfc790d62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:16:35.558602 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bc4aa9-e681-48b2-b8b1-5e7bfc790d62 service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:16:35.579292 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1291/2585] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:16:35 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:16:45.576672 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=38 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:17:15.592461 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5128b3-81b4-4dc1-9296-8d1466df00d3 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:17:15.613228 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5128b3-81b4-4dc1-9296-8d1466df00d3 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:17:15.617625 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685fe44a-88e6-4f1c-9666-ec547518faf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:17:15.618276 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685fe44a-88e6-4f1c-9666-ec547518faf2 service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:17:15.618565 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685fe44a-88e6-4f1c-9666-ec547518faf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:17:15.618846 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685fe44a-88e6-4f1c-9666-ec547518faf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:17:15.620677 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-685fe44a-88e6-4f1c-9666-ec547518faf2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:17:15.643197 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685fe44a-88e6-4f1c-9666-ec547518faf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:17:15.644783 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685fe44a-88e6-4f1c-9666-ec547518faf2 service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:17:15.652906 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1295/2586] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:17:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:17:25.649231 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=15 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:19:30.729214 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bc4aa9-e681-48b2-b8b1-5e7bfc790d62 service cinder] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:19:30.730598 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f6bc4aa9-e681-48b2-b8b1-5e7bfc790d62 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:19:30.753024 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bc4aa9-e681-48b2-b8b1-5e7bfc790d62 service cinder] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:19:30.756602 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6933bcca-bc9e-438f-a249-bbbf8e89d445 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:19:30.757233 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6933bcca-bc9e-438f-a249-bbbf8e89d445 service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:19:30.757580 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6933bcca-bc9e-438f-a249-bbbf8e89d445 service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:19:30.757778 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6933bcca-bc9e-438f-a249-bbbf8e89d445 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:19:30.759550 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-6933bcca-bc9e-438f-a249-bbbf8e89d445 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:19:30.817790 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6933bcca-bc9e-438f-a249-bbbf8e89d445 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:19:30.819223 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6933bcca-bc9e-438f-a249-bbbf8e89d445 service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:19:30.828957 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1292/2587] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:19:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:19:40.824778 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=33 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:19:50.569803 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685fe44a-88e6-4f1c-9666-ec547518faf2 service neutron] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:19:50.579362 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685fe44a-88e6-4f1c-9666-ec547518faf2 service neutron] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:19:50.582958 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64ed324-f555-42ef-9140-c254ae9ad64e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:19:50.583642 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ed324-f555-42ef-9140-c254ae9ad64e service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:19:50.583929 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ed324-f555-42ef-9140-c254ae9ad64e service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:19:50.584206 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ed324-f555-42ef-9140-c254ae9ad64e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:19:50.593053 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64ed324-f555-42ef-9140-c254ae9ad64e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:19:50.594555 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64ed324-f555-42ef-9140-c254ae9ad64e service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:19:50.603992 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1296/2588] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:19:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:20:00.599403 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=6 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:20:21.835951 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6933bcca-bc9e-438f-a249-bbbf8e89d445 service cinder] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:20:21.847561 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6933bcca-bc9e-438f-a249-bbbf8e89d445 service cinder] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:20:21.854440 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c794bcf1-967a-436b-b688-8608a093410a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70f704a231a5474e8067a6c2dd88b355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:20:21.854440 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c794bcf1-967a-436b-b688-8608a093410a service placement] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:20:21.854440 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c794bcf1-967a-436b-b688-8608a093410a service placement] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:20:21.854440 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c794bcf1-967a-436b-b688-8608a093410a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:20:21.862914 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c794bcf1-967a-436b-b688-8608a093410a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:20:21.864535 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c794bcf1-967a-436b-b688-8608a093410a service placement] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:20:21.875173 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1293/2589] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:20:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:20:31.555951 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64ed324-f555-42ef-9140-c254ae9ad64e service cinder] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:20:31.565428 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-c64ed324-f555-42ef-9140-c254ae9ad64e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:20:31.623314 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c794bcf1-967a-436b-b688-8608a093410a service placement] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:20:31.633216 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c794bcf1-967a-436b-b688-8608a093410a service placement] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:20:31.637534 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21001e5-a5bd-494a-9037-5de740addf6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:20:31.638520 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21001e5-a5bd-494a-9037-5de740addf6c service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:20:31.638520 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21001e5-a5bd-494a-9037-5de740addf6c service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:20:31.638872 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21001e5-a5bd-494a-9037-5de740addf6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:20:31.640723 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f21001e5-a5bd-494a-9037-5de740addf6c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:20:31.648110 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64ed324-f555-42ef-9140-c254ae9ad64e service cinder] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:20:31.652061 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2beaaa-ee58-4d04-816e-76a13ad66727 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0887908bcb0d4072bf6acb4cbaf1af83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:20:31.652724 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2beaaa-ee58-4d04-816e-76a13ad66727 service glance] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:20:31.653014 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2beaaa-ee58-4d04-816e-76a13ad66727 service glance] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:20:31.653292 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2beaaa-ee58-4d04-816e-76a13ad66727 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:20:31.655229 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-6a2beaaa-ee58-4d04-816e-76a13ad66727 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:20:31.670724 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application [None req-6a2beaaa-ee58-4d04-816e-76a13ad66727 service glance] Could not find project: aaeb16bbae374566926cf8512b00fc73.: keystone.exception.ProjectNotFound: Could not find project: aaeb16bbae374566926cf8512b00fc73. Jul 18 17:20:31.670724 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 17:20:31.670724 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 17:20:31.670724 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 17:20:31.670724 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 17:20:31.670724 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 17:20:31.670724 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 17:20:31.670724 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 17:20:31.670724 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 17:20:31.670724 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 17:20:31.670724 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 17:20:31.670724 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 17:20:31.670724 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Jul 18 17:20:31.670724 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Jul 18 17:20:31.670724 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Jul 18 17:20:31.671939 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Jul 18 17:20:31.671939 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Jul 18 17:20:31.671939 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Jul 18 17:20:31.671939 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 17:20:31.671939 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 17:20:31.671939 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Jul 18 17:20:31.671939 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Jul 18 17:20:31.671939 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 17:20:31.671939 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 17:20:31.671939 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 17:20:31.671939 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 17:20:31.671939 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 17:20:31.671939 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 17:20:31.671939 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 17:20:31.671939 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 17:20:31.671939 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 17:20:31.671939 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 17:20:31.673285 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 17:20:31.673285 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 17:20:31.673285 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 17:20:31.673285 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 17:20:31.673285 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Jul 18 17:20:31.673285 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Jul 18 17:20:31.673285 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Jul 18 17:20:31.673285 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application self._validate_token_resources() Jul 18 17:20:31.673285 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Jul 18 17:20:31.673285 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Jul 18 17:20:31.673285 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Jul 18 17:20:31.673285 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Jul 18 17:20:31.673285 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 17:20:31.673285 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 17:20:31.673285 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 17:20:31.673285 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 17:20:31.673285 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 17:20:31.674779 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 17:20:31.674779 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 17:20:31.674779 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application with Lock( Jul 18 17:20:31.674779 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 17:20:31.674779 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._enter() Jul 18 17:20:31.674779 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 17:20:31.674779 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 17:20:31.674779 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 17:20:31.674779 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.creator() Jul 18 17:20:31.674779 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 17:20:31.674779 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application created_value = creator( Jul 18 17:20:31.674779 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 17:20:31.674779 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 17:20:31.674779 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 17:20:31.674779 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 17:20:31.674779 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 17:20:31.674779 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 17:20:31.676023 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: aaeb16bbae374566926cf8512b00fc73. Jul 18 17:20:31.676023 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: ERROR keystone.server.flask.application Jul 18 17:20:31.676023 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1297/2590] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:20:31 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 117 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Jul 18 17:20:31.681155 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21001e5-a5bd-494a-9037-5de740addf6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:20:31.682755 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21001e5-a5bd-494a-9037-5de740addf6c service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:20:31.703711 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1294/2591] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:20:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 17:20:37.460573 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1edf75-f65f-4e02-b399-9263d416aa89 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:20:37.461052 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1edf75-f65f-4e02-b399-9263d416aa89 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:20:37.461164 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1edf75-f65f-4e02-b399-9263d416aa89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:20:37.508130 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-de1edf75-f65f-4e02-b399-9263d416aa89 None None] MFA Rules not processed for user `a9452b2e7f334f0387ecfe7194a7b38b`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:20:37.509998 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-de1edf75-f65f-4e02-b399-9263d416aa89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:20:37.526389 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1298/2592] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:20:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:20:37.753533 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21001e5-a5bd-494a-9037-5de740addf6c service cinder] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:20:37.763651 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21001e5-a5bd-494a-9037-5de740addf6c service cinder] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:20:37.767145 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65534757-20c3-4ffc-b995-2a72966e8d7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:20:37.767824 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65534757-20c3-4ffc-b995-2a72966e8d7e service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:20:37.768110 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65534757-20c3-4ffc-b995-2a72966e8d7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:20:37.768388 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65534757-20c3-4ffc-b995-2a72966e8d7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:20:37.776371 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65534757-20c3-4ffc-b995-2a72966e8d7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:20:37.777807 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65534757-20c3-4ffc-b995-2a72966e8d7e service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:20:37.785661 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1295/2593] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:20:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:20:38.184233 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1edf75-f65f-4e02-b399-9263d416aa89 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:20:38.196709 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1edf75-f65f-4e02-b399-9263d416aa89 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:20:38.201726 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f394138-9fc3-4540-9186-adc37cfeb590 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70f704a231a5474e8067a6c2dd88b355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:20:38.202378 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f394138-9fc3-4540-9186-adc37cfeb590 service placement] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:20:38.202663 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f394138-9fc3-4540-9186-adc37cfeb590 service placement] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:20:38.202942 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f394138-9fc3-4540-9186-adc37cfeb590 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:20:38.212479 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f394138-9fc3-4540-9186-adc37cfeb590 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:20:38.214055 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f394138-9fc3-4540-9186-adc37cfeb590 service placement] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:20:38.225018 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1299/2594] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:20:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:20:42.181296 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65534757-20c3-4ffc-b995-2a72966e8d7e service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:20:42.194231 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65534757-20c3-4ffc-b995-2a72966e8d7e service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:20:42.200236 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c99eeeb-d567-4bf3-8aa8-8dc77f2acc5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:20:42.200871 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99eeeb-d567-4bf3-8aa8-8dc77f2acc5c service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:20:42.201161 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99eeeb-d567-4bf3-8aa8-8dc77f2acc5c service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:20:42.201442 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99eeeb-d567-4bf3-8aa8-8dc77f2acc5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:20:42.203260 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-5c99eeeb-d567-4bf3-8aa8-8dc77f2acc5c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:20:42.216082 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c99eeeb-d567-4bf3-8aa8-8dc77f2acc5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:20:42.217673 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c99eeeb-d567-4bf3-8aa8-8dc77f2acc5c service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:20:42.228383 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1296/2595] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:20:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:20:48.220823 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=58 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:20:49.000221 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f394138-9fc3-4540-9186-adc37cfeb590 service placement] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:20:49.012931 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f394138-9fc3-4540-9186-adc37cfeb590 service placement] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:20:49.019168 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c734a1e8-19de-419e-8525-140b45869fac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:20:49.020488 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734a1e8-19de-419e-8525-140b45869fac service nova] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:20:49.020488 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734a1e8-19de-419e-8525-140b45869fac service nova] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:20:49.020488 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734a1e8-19de-419e-8525-140b45869fac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:20:49.031041 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c734a1e8-19de-419e-8525-140b45869fac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:20:49.037927 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c734a1e8-19de-419e-8525-140b45869fac service nova] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:20:49.046151 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1300/2596] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:20:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:20:49.418576 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5ff66-ff32-4659-8d5d-c24a6c8940c4 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:20:49.424687 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5ff66-ff32-4659-8d5d-c24a6c8940c4 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:20:49.424687 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5ff66-ff32-4659-8d5d-c24a6c8940c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:20:49.477381 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-2ae5ff66-ff32-4659-8d5d-c24a6c8940c4 None None] MFA Rules not processed for user `da799b0044f54da6835aa06e43ce1a6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:20:49.478638 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-2ae5ff66-ff32-4659-8d5d-c24a6c8940c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:20:49.543341 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1297/2597] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jul 18 17:20:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:20:49.682949 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c734a1e8-19de-419e-8525-140b45869fac service nova] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:20:49.684427 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-c734a1e8-19de-419e-8525-140b45869fac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:20:49.775835 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c734a1e8-19de-419e-8525-140b45869fac service nova] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:20:49.780987 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71229ed-4083-4468-bee5-302580a34643 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43854d7a6df2466eb7976fabe6011767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:20:49.781827 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71229ed-4083-4468-bee5-302580a34643 service swift] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:20:49.782119 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71229ed-4083-4468-bee5-302580a34643 service swift] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:20:49.782401 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71229ed-4083-4468-bee5-302580a34643 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:20:49.791661 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71229ed-4083-4468-bee5-302580a34643 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:20:49.793243 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71229ed-4083-4468-bee5-302580a34643 service swift] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:20:49.801981 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1301/2598] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jul 18 17:20:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 17:20:59.539897 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=80 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:20:59.798407 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=47 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:21:08.216762 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320dd73b-df77-44ba-a043-294c12426fc1 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:21:08.217320 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320dd73b-df77-44ba-a043-294c12426fc1 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:21:08.217320 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320dd73b-df77-44ba-a043-294c12426fc1 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:21:08.218888 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1298/2599] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jul 18 17:21:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 17:21:08.224788 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5c0d3e-fa26-42e0-acf9-9d89724e4132 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:21:08.225278 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5c0d3e-fa26-42e0-acf9-9d89724e4132 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:21:08.225649 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5c0d3e-fa26-42e0-acf9-9d89724e4132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:21:08.265229 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-ff5c0d3e-fa26-42e0-acf9-9d89724e4132 None None] MFA Rules not processed for user `5b868bfa44b94ca5b470166d39f20e54`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:21:08.266484 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-ff5c0d3e-fa26-42e0-acf9-9d89724e4132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:21:08.276661 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1302/2600] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jul 18 17:21:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 17:21:08.290560 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320dd73b-df77-44ba-a043-294c12426fc1 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:21:08.292161 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-320dd73b-df77-44ba-a043-294c12426fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:21:08.306196 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320dd73b-df77-44ba-a043-294c12426fc1 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:21:08.309803 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72939440-0771-46fb-8223-9e35235654b4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0887908bcb0d4072bf6acb4cbaf1af83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:21:08.310439 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72939440-0771-46fb-8223-9e35235654b4 service glance] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:21:08.310728 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72939440-0771-46fb-8223-9e35235654b4 service glance] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:21:08.311033 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72939440-0771-46fb-8223-9e35235654b4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:21:08.321878 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72939440-0771-46fb-8223-9e35235654b4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:21:08.323303 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72939440-0771-46fb-8223-9e35235654b4 service glance] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:21:08.333989 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1299/2601] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:21:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:21:08.564687 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad698c9e-8e84-4696-9067-57fe976df8bc None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:21:08.564903 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad698c9e-8e84-4696-9067-57fe976df8bc None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:21:08.565158 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad698c9e-8e84-4696-9067-57fe976df8bc None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:21:08.566538 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1303/2602] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jul 18 17:21:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 17:21:08.571671 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89db9f6-4a04-4201-9852-5049b5405762 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:21:08.572143 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89db9f6-4a04-4201-9852-5049b5405762 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:21:08.572429 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89db9f6-4a04-4201-9852-5049b5405762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:21:08.609265 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-c89db9f6-4a04-4201-9852-5049b5405762 None None] MFA Rules not processed for user `7a6054dbfbbc487794d2a32061b0ac82`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:21:08.610570 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-c89db9f6-4a04-4201-9852-5049b5405762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:21:08.617663 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1300/2603] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jul 18 17:21:08 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:21:08.629755 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad698c9e-8e84-4696-9067-57fe976df8bc None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:21:08.640943 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad698c9e-8e84-4696-9067-57fe976df8bc None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:21:08.644474 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eac7a67-13cb-469c-bafd-b93c41e4249a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:21:08.645112 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac7a67-13cb-469c-bafd-b93c41e4249a service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:21:08.645401 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac7a67-13cb-469c-bafd-b93c41e4249a service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:21:08.645681 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac7a67-13cb-469c-bafd-b93c41e4249a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:21:08.653145 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eac7a67-13cb-469c-bafd-b93c41e4249a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:21:08.654550 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eac7a67-13cb-469c-bafd-b93c41e4249a service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:21:08.662283 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1304/2604] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:21:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:21:08.746210 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65272e23-c158-4500-a4d9-f145def612d2 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:21:08.746506 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65272e23-c158-4500-a4d9-f145def612d2 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:21:08.746790 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65272e23-c158-4500-a4d9-f145def612d2 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:21:08.748232 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1301/2605] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jul 18 17:21:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 17:21:08.753120 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9585f1b-b7f1-421f-891d-2f7797db298d None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:21:08.753412 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9585f1b-b7f1-421f-891d-2f7797db298d None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:21:08.753691 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9585f1b-b7f1-421f-891d-2f7797db298d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:21:08.771936 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-b9585f1b-b7f1-421f-891d-2f7797db298d None None] MFA Rules not processed for user `5b868bfa44b94ca5b470166d39f20e54`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:21:08.773719 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-b9585f1b-b7f1-421f-891d-2f7797db298d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:21:08.783185 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1305/2606] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jul 18 17:21:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 17:21:08.794146 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65272e23-c158-4500-a4d9-f145def612d2 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:21:08.805354 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65272e23-c158-4500-a4d9-f145def612d2 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:21:08.808988 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b37bbc7-c26c-4c6d-87fc-9b58e24ca663 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:21:08.809617 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b37bbc7-c26c-4c6d-87fc-9b58e24ca663 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:21:08.809904 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b37bbc7-c26c-4c6d-87fc-9b58e24ca663 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:21:08.810184 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b37bbc7-c26c-4c6d-87fc-9b58e24ca663 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:21:08.820767 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b37bbc7-c26c-4c6d-87fc-9b58e24ca663 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:21:08.822148 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b37bbc7-c26c-4c6d-87fc-9b58e24ca663 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:21:08.833270 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1302/2607] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:21:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:21:12.969299 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71b61c-e3ed-469f-9d4e-5cf09e380ab0 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:21:12.969803 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71b61c-e3ed-469f-9d4e-5cf09e380ab0 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:21:12.969918 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71b61c-e3ed-469f-9d4e-5cf09e380ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:21:12.989261 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-ba71b61c-e3ed-469f-9d4e-5cf09e380ab0 None None] MFA Rules not processed for user `da799b0044f54da6835aa06e43ce1a6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:21:12.990681 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-ba71b61c-e3ed-469f-9d4e-5cf09e380ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:21:12.998648 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1306/2608] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jul 18 17:21:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:21:13.011020 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b37bbc7-c26c-4c6d-87fc-9b58e24ca663 service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:21:13.022616 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b37bbc7-c26c-4c6d-87fc-9b58e24ca663 service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:21:13.028207 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99fee61-2de6-4b46-9166-e4472ad46643 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43854d7a6df2466eb7976fabe6011767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:21:13.033162 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99fee61-2de6-4b46-9166-e4472ad46643 service swift] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:21:13.033162 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99fee61-2de6-4b46-9166-e4472ad46643 service swift] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:21:13.033162 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99fee61-2de6-4b46-9166-e4472ad46643 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:21:13.040763 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99fee61-2de6-4b46-9166-e4472ad46643 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:21:13.042339 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99fee61-2de6-4b46-9166-e4472ad46643 service swift] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:21:13.051511 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1303/2609] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jul 18 17:21:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 17:21:14.092767 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba71b61c-e3ed-469f-9d4e-5cf09e380ab0 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:21:14.106520 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba71b61c-e3ed-469f-9d4e-5cf09e380ab0 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:21:14.117080 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a109902-25ae-4b76-831f-9e89772f9df9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:21:14.118096 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a109902-25ae-4b76-831f-9e89772f9df9 service nova] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:21:14.118707 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a109902-25ae-4b76-831f-9e89772f9df9 service nova] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:21:14.119073 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a109902-25ae-4b76-831f-9e89772f9df9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:21:14.121342 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-9a109902-25ae-4b76-831f-9e89772f9df9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:21:14.136498 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a109902-25ae-4b76-831f-9e89772f9df9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:21:14.138121 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a109902-25ae-4b76-831f-9e89772f9df9 service nova] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:21:14.170815 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1307/2610] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:21:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:21:23.047539 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=27 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:21:24.163058 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=27 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:21:35.385111 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99fee61-2de6-4b46-9166-e4472ad46643 service swift] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:21:35.394422 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99fee61-2de6-4b46-9166-e4472ad46643 service swift] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:21:35.398033 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08873349-1485-45e0-b11c-9d0623c013b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:21:35.398675 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08873349-1485-45e0-b11c-9d0623c013b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:21:35.399064 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08873349-1485-45e0-b11c-9d0623c013b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:21:35.399346 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08873349-1485-45e0-b11c-9d0623c013b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:21:35.407792 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08873349-1485-45e0-b11c-9d0623c013b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:21:35.409211 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08873349-1485-45e0-b11c-9d0623c013b0 service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:21:35.418859 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1304/2611] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:21:35 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:21:39.577983 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74664ce2-c92d-40c7-bc5d-8c233ac33d84 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:21:39.578441 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74664ce2-c92d-40c7-bc5d-8c233ac33d84 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:21:39.578540 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74664ce2-c92d-40c7-bc5d-8c233ac33d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:21:39.596288 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-74664ce2-c92d-40c7-bc5d-8c233ac33d84 None None] MFA Rules not processed for user `da799b0044f54da6835aa06e43ce1a6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:21:39.597496 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-74664ce2-c92d-40c7-bc5d-8c233ac33d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:21:39.605072 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1308/2612] 10.222.0.26 () {58 vars in 967 bytes} [Mon Jul 18 17:21:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:21:39.617761 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08873349-1485-45e0-b11c-9d0623c013b0 service cinder] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:21:39.631095 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08873349-1485-45e0-b11c-9d0623c013b0 service cinder] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:21:39.638604 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a8203-7b52-48f4-8ff6-0a8f6a5fa3f5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43854d7a6df2466eb7976fabe6011767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:21:39.639968 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a8203-7b52-48f4-8ff6-0a8f6a5fa3f5 service swift] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:21:39.640262 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a8203-7b52-48f4-8ff6-0a8f6a5fa3f5 service swift] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:21:39.640542 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a8203-7b52-48f4-8ff6-0a8f6a5fa3f5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:21:39.649485 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312a8203-7b52-48f4-8ff6-0a8f6a5fa3f5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:21:39.651230 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312a8203-7b52-48f4-8ff6-0a8f6a5fa3f5 service swift] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:21:39.660407 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1305/2613] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jul 18 17:21:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 17:21:49.591543 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=3 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:21:49.656275 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=12 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:21:58.449572 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74664ce2-c92d-40c7-bc5d-8c233ac33d84 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:21:58.459017 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74664ce2-c92d-40c7-bc5d-8c233ac33d84 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:21:58.462586 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebf60f5-51e5-4890-a412-607668265710 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:21:58.463225 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf60f5-51e5-4890-a412-607668265710 service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:21:58.463575 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf60f5-51e5-4890-a412-607668265710 service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:21:58.463860 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf60f5-51e5-4890-a412-607668265710 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:21:58.465618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-4ebf60f5-51e5-4890-a412-607668265710 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:21:58.498505 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebf60f5-51e5-4890-a412-607668265710 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:21:58.500074 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebf60f5-51e5-4890-a412-607668265710 service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:21:58.519064 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1309/2614] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:21:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:21:59.030321 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a8203-7b52-48f4-8ff6-0a8f6a5fa3f5 service swift] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:21:59.041605 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a8203-7b52-48f4-8ff6-0a8f6a5fa3f5 service swift] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:21:59.047249 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d605f8-0cf0-4f08-8e9e-60470166ac60 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70f704a231a5474e8067a6c2dd88b355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:21:59.047925 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d605f8-0cf0-4f08-8e9e-60470166ac60 service placement] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:21:59.048198 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d605f8-0cf0-4f08-8e9e-60470166ac60 service placement] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:21:59.048469 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d605f8-0cf0-4f08-8e9e-60470166ac60 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:21:59.050502 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d6d605f8-0cf0-4f08-8e9e-60470166ac60 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:21:59.062311 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d605f8-0cf0-4f08-8e9e-60470166ac60 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:21:59.063937 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d605f8-0cf0-4f08-8e9e-60470166ac60 service placement] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:21:59.077558 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1306/2615] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:21:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:21:59.147562 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebf60f5-51e5-4890-a412-607668265710 service neutron] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:21:59.149021 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-4ebf60f5-51e5-4890-a412-607668265710 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:21:59.162127 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebf60f5-51e5-4890-a412-607668265710 service neutron] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:21:59.167458 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786697fb-3b09-4ab9-ac2e-e4a5cbab65f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:21:59.168450 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786697fb-3b09-4ab9-ac2e-e4a5cbab65f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:21:59.168738 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786697fb-3b09-4ab9-ac2e-e4a5cbab65f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:21:59.169016 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786697fb-3b09-4ab9-ac2e-e4a5cbab65f1 admin admin] PATH_INFO: `/v3/users/a9452b2e7f334f0387ecfe7194a7b38b` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:21:59.170253 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786697fb-3b09-4ab9-ac2e-e4a5cbab65f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9452b2e7f334f0387ecfe7194a7b38b)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:21:59.172280 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786697fb-3b09-4ab9-ac2e-e4a5cbab65f1 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:21:59.370658 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-786697fb-3b09-4ab9-ac2e-e4a5cbab65f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9452b2e7f334f0387ecfe7194a7b38b'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:21:59.436826 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-786697fb-3b09-4ab9-ac2e-e4a5cbab65f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9452b2e7f334f0387ecfe7194a7b38b'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:21:59.437450 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-786697fb-3b09-4ab9-ac2e-e4a5cbab65f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9452b2e7f334f0387ecfe7194a7b38b'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:21:59.438061 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-786697fb-3b09-4ab9-ac2e-e4a5cbab65f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9452b2e7f334f0387ecfe7194a7b38b'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:21:59.445544 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-786697fb-3b09-4ab9-ac2e-e4a5cbab65f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9452b2e7f334f0387ecfe7194a7b38b'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:21:59.463107 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1310/2616] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 17:21:59 2022] DELETE /identity/v3/users/a9452b2e7f334f0387ecfe7194a7b38b => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:21:59.713238 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d605f8-0cf0-4f08-8e9e-60470166ac60 service placement] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:21:59.724344 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d605f8-0cf0-4f08-8e9e-60470166ac60 service placement] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:21:59.785991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd007c45-a24d-4c9a-a9a8-08e33c635f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:21:59.786850 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd007c45-a24d-4c9a-a9a8-08e33c635f84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:21:59.787133 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd007c45-a24d-4c9a-a9a8-08e33c635f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:21:59.787408 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd007c45-a24d-4c9a-a9a8-08e33c635f84 admin admin] PATH_INFO: `/v3/projects/8368f31817254dafaea912485f3aa86b` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:21:59.788630 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd007c45-a24d-4c9a-a9a8-08e33c635f84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8368f31817254dafaea912485f3aa86b)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:21:59.790543 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd007c45-a24d-4c9a-a9a8-08e33c635f84 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:21:59.888621 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-cd007c45-a24d-4c9a-a9a8-08e33c635f84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8368f31817254dafaea912485f3aa86b', 'request_id': 'req-cd007c45-a24d-4c9a-a9a8-08e33c635f84'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:21:59.894200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-cd007c45-a24d-4c9a-a9a8-08e33c635f84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8368f31817254dafaea912485f3aa86b', 'request_id': 'req-cd007c45-a24d-4c9a-a9a8-08e33c635f84'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:21:59.900704 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-cd007c45-a24d-4c9a-a9a8-08e33c635f84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8368f31817254dafaea912485f3aa86b', 'request_id': 'req-cd007c45-a24d-4c9a-a9a8-08e33c635f84'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:21:59.965089 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-cd007c45-a24d-4c9a-a9a8-08e33c635f84 admin admin] The token cache is being invalidate because project 8368f31817254dafaea912485f3aa86b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 17:21:59.965317 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-cd007c45-a24d-4c9a-a9a8-08e33c635f84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:21:59.966009 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-cd007c45-a24d-4c9a-a9a8-08e33c635f84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:21:59.967625 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1307/2617] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 17:21:59 2022] DELETE /identity/v3/projects/8368f31817254dafaea912485f3aa86b => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:21:59.972848 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533ebb4-014e-4a0f-8c30-d2b29c55de2e None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:21:59.973140 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533ebb4-014e-4a0f-8c30-d2b29c55de2e None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:21:59.973423 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533ebb4-014e-4a0f-8c30-d2b29c55de2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:22:00.001367 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-2533ebb4-014e-4a0f-8c30-d2b29c55de2e None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:22:00.002598 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-2533ebb4-014e-4a0f-8c30-d2b29c55de2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:22:00.067730 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1311/2618] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:21:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:22:00.075349 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a22ff1-3d96-493a-a9f7-5a296cb10403 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:22:00.075875 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a22ff1-3d96-493a-a9f7-5a296cb10403 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:22:00.075988 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a22ff1-3d96-493a-a9f7-5a296cb10403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:22:00.094038 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-73a22ff1-3d96-493a-a9f7-5a296cb10403 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:22:00.095241 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-73a22ff1-3d96-493a-a9f7-5a296cb10403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:22:00.103188 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1308/2619] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:22:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:22:00.107976 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533ebb4-014e-4a0f-8c30-d2b29c55de2e None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:22:00.117772 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533ebb4-014e-4a0f-8c30-d2b29c55de2e None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:22:00.123341 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eacaf0-1074-4e0e-a0c0-41f058634a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:22:00.124065 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eacaf0-1074-4e0e-a0c0-41f058634a6f admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:22:00.124350 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eacaf0-1074-4e0e-a0c0-41f058634a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:22:00.124632 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eacaf0-1074-4e0e-a0c0-41f058634a6f admin admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:22:00.125363 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eacaf0-1074-4e0e-a0c0-41f058634a6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 17:22:00.125698 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eacaf0-1074-4e0e-a0c0-41f058634a6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:22:00.127307 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eacaf0-1074-4e0e-a0c0-41f058634a6f admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:22:00.139185 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1312/2620] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 17:22:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 17:22:00.143320 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a22ff1-3d96-493a-a9f7-5a296cb10403 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:22:00.152851 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a22ff1-3d96-493a-a9f7-5a296cb10403 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:22:00.157453 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2447c2-cecc-431c-a546-52d231744a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:22:00.158900 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2447c2-cecc-431c-a546-52d231744a33 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:22:00.158900 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2447c2-cecc-431c-a546-52d231744a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:22:00.158900 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2447c2-cecc-431c-a546-52d231744a33 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:22:00.159357 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2447c2-cecc-431c-a546-52d231744a33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:22:00.161266 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2447c2-cecc-431c-a546-52d231744a33 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:22:00.232519 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1309/2621] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 17:22:00 2022] POST /identity/v3/projects => generated 385 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:22:00.236532 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eacaf0-1074-4e0e-a0c0-41f058634a6f admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:22:00.247044 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eacaf0-1074-4e0e-a0c0-41f058634a6f admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:22:00.254724 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:22:00.303170 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a8ec6c-f501-4e69-9337-2fd18925bbb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:22:00.303885 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8ec6c-f501-4e69-9337-2fd18925bbb9 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:22:00.304165 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8ec6c-f501-4e69-9337-2fd18925bbb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:22:00.304439 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8ec6c-f501-4e69-9337-2fd18925bbb9 admin admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:22:00.305125 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a8ec6c-f501-4e69-9337-2fd18925bbb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:22:00.307029 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a8ec6c-f501-4e69-9337-2fd18925bbb9 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:22:00.380515 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1313/2622] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 17:22:00 2022] POST /identity/v3/users => generated 293 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:22:00.384625 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2447c2-cecc-431c-a546-52d231744a33 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:22:00.395053 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2447c2-cecc-431c-a546-52d231744a33 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:22:00.400765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72faf060-0983-43c4-9b1a-ed4c5af61848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:22:00.401904 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72faf060-0983-43c4-9b1a-ed4c5af61848 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:22:00.402185 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72faf060-0983-43c4-9b1a-ed4c5af61848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:22:00.402457 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72faf060-0983-43c4-9b1a-ed4c5af61848 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:22:00.403176 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72faf060-0983-43c4-9b1a-ed4c5af61848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:22:00.404822 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72faf060-0983-43c4-9b1a-ed4c5af61848 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:22:00.415334 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1310/2623] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 17:22:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:22:00.420044 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a8ec6c-f501-4e69-9337-2fd18925bbb9 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:22:00.430532 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a8ec6c-f501-4e69-9337-2fd18925bbb9 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:22:00.435204 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6978de7f-80ed-4a7a-a938-783d243f25ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:22:00.435886 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978de7f-80ed-4a7a-a938-783d243f25ae admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:22:00.436166 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978de7f-80ed-4a7a-a938-783d243f25ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:22:00.436439 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978de7f-80ed-4a7a-a938-783d243f25ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:22:00.437156 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6978de7f-80ed-4a7a-a938-783d243f25ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:22:00.438731 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6978de7f-80ed-4a7a-a938-783d243f25ae admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:22:00.448692 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1314/2624] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 17:22:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:22:00.452795 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72faf060-0983-43c4-9b1a-ed4c5af61848 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:22:00.462582 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72faf060-0983-43c4-9b1a-ed4c5af61848 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:22:00.467765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a4d8a1-dc8b-41bd-84c9-ef1bbc662219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:22:00.468628 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4d8a1-dc8b-41bd-84c9-ef1bbc662219 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:22:00.468907 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4d8a1-dc8b-41bd-84c9-ef1bbc662219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:22:00.469180 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4d8a1-dc8b-41bd-84c9-ef1bbc662219 admin admin] PATH_INFO: `/v3/projects/0d565c1992f5420fa5166d2cfb6f1ab2/users/a6ebcad94c6048c8b174cfcee63a15a7/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:22:00.483839 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a4d8a1-dc8b-41bd-84c9-ef1bbc662219 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d565c1992f5420fa5166d2cfb6f1ab2, user_id=a6ebcad94c6048c8b174cfcee63a15a7, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:22:00.486388 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a4d8a1-dc8b-41bd-84c9-ef1bbc662219 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:22:00.562486 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1311/2625] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 17:22:00 2022] PUT /identity/v3/projects/0d565c1992f5420fa5166d2cfb6f1ab2/users/a6ebcad94c6048c8b174cfcee63a15a7/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 17:22:00.567750 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39058e1-ce34-4926-b07b-4e131b6edbf0 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:22:00.568227 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39058e1-ce34-4926-b07b-4e131b6edbf0 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:22:00.568329 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39058e1-ce34-4926-b07b-4e131b6edbf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:22:00.587348 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-f39058e1-ce34-4926-b07b-4e131b6edbf0 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:22:00.593354 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-f39058e1-ce34-4926-b07b-4e131b6edbf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:22:00.647015 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1315/2626] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:22:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:22:00.657189 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a4d8a1-dc8b-41bd-84c9-ef1bbc662219 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:22:00.660857 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-64a4d8a1-dc8b-41bd-84c9-ef1bbc662219 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:22:00.673591 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:22:00.717595 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a4d8a1-dc8b-41bd-84c9-ef1bbc662219 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:22:00.721522 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f3dce8-6e45-433c-9ad8-a93b7784a4f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:22:00.722180 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3dce8-6e45-433c-9ad8-a93b7784a4f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:22:00.722461 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3dce8-6e45-433c-9ad8-a93b7784a4f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:22:00.722737 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3dce8-6e45-433c-9ad8-a93b7784a4f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:22:00.733185 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f3dce8-6e45-433c-9ad8-a93b7784a4f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:22:00.734682 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f3dce8-6e45-433c-9ad8-a93b7784a4f0 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:22:00.744559 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1312/2627] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:22:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:22:05.943910 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65538b24-28e1-4d86-bf1a-6443e78faabb None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:22:05.944444 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65538b24-28e1-4d86-bf1a-6443e78faabb None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:22:05.944535 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65538b24-28e1-4d86-bf1a-6443e78faabb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:22:05.975603 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-65538b24-28e1-4d86-bf1a-6443e78faabb None None] MFA Rules not processed for user `a6ebcad94c6048c8b174cfcee63a15a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:22:05.976979 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-65538b24-28e1-4d86-bf1a-6443e78faabb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:22:06.052133 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1316/2628] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:22:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:22:06.060498 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f3dce8-6e45-433c-9ad8-a93b7784a4f0 service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:22:06.070302 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f3dce8-6e45-433c-9ad8-a93b7784a4f0 service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:22:06.075639 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5246f7a5-b17b-4607-b239-52fad7d65f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:22:06.076291 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5246f7a5-b17b-4607-b239-52fad7d65f09 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:22:06.076581 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5246f7a5-b17b-4607-b239-52fad7d65f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:22:06.081988 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5246f7a5-b17b-4607-b239-52fad7d65f09 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:22:06.081988 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5246f7a5-b17b-4607-b239-52fad7d65f09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:22:06.081988 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5246f7a5-b17b-4607-b239-52fad7d65f09 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:22:06.158881 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1313/2629] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 17:22:06 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:22:06.163187 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65538b24-28e1-4d86-bf1a-6443e78faabb None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:22:06.174020 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65538b24-28e1-4d86-bf1a-6443e78faabb None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:22:06.190748 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:22:06.232718 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135e3c11-19c2-47dd-a5d1-3ebd677c8425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:22:06.233341 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e3c11-19c2-47dd-a5d1-3ebd677c8425 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:22:06.233627 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e3c11-19c2-47dd-a5d1-3ebd677c8425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:22:06.233902 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e3c11-19c2-47dd-a5d1-3ebd677c8425 admin admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:22:06.234596 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135e3c11-19c2-47dd-a5d1-3ebd677c8425 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:22:06.236563 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135e3c11-19c2-47dd-a5d1-3ebd677c8425 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:22:06.311152 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1317/2630] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 17:22:06 2022] POST /identity/v3/users => generated 299 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:22:06.315276 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5246f7a5-b17b-4607-b239-52fad7d65f09 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:22:06.326208 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5246f7a5-b17b-4607-b239-52fad7d65f09 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:22:06.330738 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf8f416-f7e1-4f60-a1eb-6a1414a46580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:22:06.331397 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf8f416-f7e1-4f60-a1eb-6a1414a46580 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:22:06.331719 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf8f416-f7e1-4f60-a1eb-6a1414a46580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:22:06.331998 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf8f416-f7e1-4f60-a1eb-6a1414a46580 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:22:06.332764 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf8f416-f7e1-4f60-a1eb-6a1414a46580 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:22:06.334390 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf8f416-f7e1-4f60-a1eb-6a1414a46580 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:22:06.345578 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1314/2631] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 17:22:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:22:06.349595 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135e3c11-19c2-47dd-a5d1-3ebd677c8425 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:22:06.359437 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135e3c11-19c2-47dd-a5d1-3ebd677c8425 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:22:06.364039 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d636abd-659d-4902-b502-5c34cfd4871b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:22:06.364913 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d636abd-659d-4902-b502-5c34cfd4871b admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:22:06.365191 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d636abd-659d-4902-b502-5c34cfd4871b admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:22:06.365465 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d636abd-659d-4902-b502-5c34cfd4871b admin admin] PATH_INFO: `/v3/projects/ec70c8db318d4f37b21bea763ebd39cc/users/c8988216d1bc4f7186900a38724e2720/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:22:06.380100 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d636abd-659d-4902-b502-5c34cfd4871b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec70c8db318d4f37b21bea763ebd39cc, user_id=c8988216d1bc4f7186900a38724e2720, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:22:06.382852 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d636abd-659d-4902-b502-5c34cfd4871b admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:22:06.451070 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1318/2632] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 17:22:06 2022] PUT /identity/v3/projects/ec70c8db318d4f37b21bea763ebd39cc/users/c8988216d1bc4f7186900a38724e2720/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:22:06.455390 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf8f416-f7e1-4f60-a1eb-6a1414a46580 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:22:06.466124 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf8f416-f7e1-4f60-a1eb-6a1414a46580 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:22:06.535086 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25476003-447f-4810-8dc6-5835a1f30d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:22:06.535815 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25476003-447f-4810-8dc6-5835a1f30d14 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:22:06.536104 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25476003-447f-4810-8dc6-5835a1f30d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:22:06.536392 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25476003-447f-4810-8dc6-5835a1f30d14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:22:06.537167 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25476003-447f-4810-8dc6-5835a1f30d14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:22:06.538787 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25476003-447f-4810-8dc6-5835a1f30d14 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:22:06.550590 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1315/2633] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 17:22:06 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:22:06.554885 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d636abd-659d-4902-b502-5c34cfd4871b admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:22:06.570665 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d636abd-659d-4902-b502-5c34cfd4871b admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:22:06.575650 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c76705-9a85-4373-b709-9be700eed4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:22:06.576554 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c76705-9a85-4373-b709-9be700eed4a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:22:06.576836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c76705-9a85-4373-b709-9be700eed4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:22:06.577111 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c76705-9a85-4373-b709-9be700eed4a2 admin admin] PATH_INFO: `/v3/projects/ec70c8db318d4f37b21bea763ebd39cc/users/c8988216d1bc4f7186900a38724e2720/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:22:06.579088 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c76705-9a85-4373-b709-9be700eed4a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec70c8db318d4f37b21bea763ebd39cc, user_id=c8988216d1bc4f7186900a38724e2720, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:22:06.581785 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c76705-9a85-4373-b709-9be700eed4a2 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:22:06.593231 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1319/2634] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 17:22:06 2022] PUT /identity/v3/projects/ec70c8db318d4f37b21bea763ebd39cc/users/c8988216d1bc4f7186900a38724e2720/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 17:22:12.074403 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cb4d1e-02d7-4628-90ac-501ef13fd92d None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:22:12.074889 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cb4d1e-02d7-4628-90ac-501ef13fd92d None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:22:12.074992 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cb4d1e-02d7-4628-90ac-501ef13fd92d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:22:12.106797 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-01cb4d1e-02d7-4628-90ac-501ef13fd92d None None] MFA Rules not processed for user `c8988216d1bc4f7186900a38724e2720`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:22:12.108643 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-01cb4d1e-02d7-4628-90ac-501ef13fd92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:22:12.186284 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1316/2635] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:22:12 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:22:12.199855 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c76705-9a85-4373-b709-9be700eed4a2 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:22:12.201631 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-47c76705-9a85-4373-b709-9be700eed4a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:22:12.263556 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c76705-9a85-4373-b709-9be700eed4a2 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:22:12.267678 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1dec41-f771-47d3-8f72-27985aa7839f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0887908bcb0d4072bf6acb4cbaf1af83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:22:12.267942 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1dec41-f771-47d3-8f72-27985aa7839f service glance] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:22:12.268227 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1dec41-f771-47d3-8f72-27985aa7839f service glance] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:22:12.268511 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1dec41-f771-47d3-8f72-27985aa7839f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:22:12.327566 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1dec41-f771-47d3-8f72-27985aa7839f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:22:12.329083 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1dec41-f771-47d3-8f72-27985aa7839f service glance] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:22:12.340032 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1320/2636] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:22:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:22:12.598712 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f208d48-85d5-4f0b-aa3b-65d7dea00279 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:22:12.599009 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f208d48-85d5-4f0b-aa3b-65d7dea00279 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:22:12.599289 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f208d48-85d5-4f0b-aa3b-65d7dea00279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:22:12.607130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:22:12.617728 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-6f208d48-85d5-4f0b-aa3b-65d7dea00279 None None] MFA Rules not processed for user `da799b0044f54da6835aa06e43ce1a6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:22:12.618995 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-6f208d48-85d5-4f0b-aa3b-65d7dea00279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:22:12.679633 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1317/2637] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jul 18 17:22:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:22:12.702400 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1dec41-f771-47d3-8f72-27985aa7839f service glance] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:22:12.703911 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-ac1dec41-f771-47d3-8f72-27985aa7839f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:22:12.747250 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:22:12.792467 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1dec41-f771-47d3-8f72-27985aa7839f service glance] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:22:12.797721 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50871188-2810-4d7e-ac87-8141d80bf711 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43854d7a6df2466eb7976fabe6011767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:22:12.798391 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50871188-2810-4d7e-ac87-8141d80bf711 service swift] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:22:12.798724 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50871188-2810-4d7e-ac87-8141d80bf711 service swift] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:22:12.799007 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50871188-2810-4d7e-ac87-8141d80bf711 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:22:12.807712 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50871188-2810-4d7e-ac87-8141d80bf711 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:22:12.809391 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50871188-2810-4d7e-ac87-8141d80bf711 service swift] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:22:12.817972 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1321/2638] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jul 18 17:22:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 17:22:18.554771 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f208d48-85d5-4f0b-aa3b-65d7dea00279 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:22:18.622989 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f208d48-85d5-4f0b-aa3b-65d7dea00279 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:22:18.629080 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac042697-763a-43db-8ccc-a1fcaf9f0234 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:22:18.630338 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac042697-763a-43db-8ccc-a1fcaf9f0234 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:22:18.630821 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac042697-763a-43db-8ccc-a1fcaf9f0234 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:22:18.631339 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac042697-763a-43db-8ccc-a1fcaf9f0234 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:22:18.634037 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-ac042697-763a-43db-8ccc-a1fcaf9f0234 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:22:18.645162 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac042697-763a-43db-8ccc-a1fcaf9f0234 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:22:18.647074 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac042697-763a-43db-8ccc-a1fcaf9f0234 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:22:18.657806 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1318/2639] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:22:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:22:22.888394 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=20 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:22:22.953240 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b4d9ac-0862-46d8-9a5a-95cf81919334 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:22:22.953623 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b4d9ac-0862-46d8-9a5a-95cf81919334 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:22:22.953909 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b4d9ac-0862-46d8-9a5a-95cf81919334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:22:22.971758 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-d4b4d9ac-0862-46d8-9a5a-95cf81919334 None None] MFA Rules not processed for user `9aec7436e3a549018fd7892969870019`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:22:22.973041 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-d4b4d9ac-0862-46d8-9a5a-95cf81919334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:22:23.029976 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1322/2640] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:22:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 17:22:24.326598 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67cfa40-47dd-41cd-94cd-063e58424975 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:22:24.327324 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67cfa40-47dd-41cd-94cd-063e58424975 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:22:24.327592 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67cfa40-47dd-41cd-94cd-063e58424975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:22:24.346759 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-b67cfa40-47dd-41cd-94cd-063e58424975 None None] MFA Rules not processed for user `a6ebcad94c6048c8b174cfcee63a15a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:22:24.348016 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-b67cfa40-47dd-41cd-94cd-063e58424975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:22:24.356245 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1319/2641] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:22:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:22:27.876928 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be0c2fb-c383-44e1-a6ad-e93cd6e7f046 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:22:27.876928 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be0c2fb-c383-44e1-a6ad-e93cd6e7f046 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:22:27.876928 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be0c2fb-c383-44e1-a6ad-e93cd6e7f046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:22:27.900787 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-0be0c2fb-c383-44e1-a6ad-e93cd6e7f046 None None] MFA Rules not processed for user `da799b0044f54da6835aa06e43ce1a6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:22:27.903104 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-0be0c2fb-c383-44e1-a6ad-e93cd6e7f046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:22:27.912917 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1323/2642] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jul 18 17:22:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:22:27.926070 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67cfa40-47dd-41cd-94cd-063e58424975 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:22:27.939174 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67cfa40-47dd-41cd-94cd-063e58424975 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:22:27.944582 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9859c96f-5177-46a3-aeb5-678e0c0c7fe1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43854d7a6df2466eb7976fabe6011767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:22:27.945369 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9859c96f-5177-46a3-aeb5-678e0c0c7fe1 service swift] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:22:27.945719 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9859c96f-5177-46a3-aeb5-678e0c0c7fe1 service swift] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:22:27.946090 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9859c96f-5177-46a3-aeb5-678e0c0c7fe1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:22:27.956479 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9859c96f-5177-46a3-aeb5-678e0c0c7fe1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:22:27.959381 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9859c96f-5177-46a3-aeb5-678e0c0c7fe1 service swift] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:22:27.991265 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1320/2643] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jul 18 17:22:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 17:22:37.889769 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=28 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:22:37.965252 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=58 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:22:57.351789 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be0c2fb-c383-44e1-a6ad-e93cd6e7f046 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:22:57.353274 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-0be0c2fb-c383-44e1-a6ad-e93cd6e7f046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:22:57.365574 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be0c2fb-c383-44e1-a6ad-e93cd6e7f046 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:22:57.369225 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76555f3-b41b-4510-93ae-36239d4374ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:22:57.369867 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76555f3-b41b-4510-93ae-36239d4374ac service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:22:57.370150 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76555f3-b41b-4510-93ae-36239d4374ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:22:57.370427 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76555f3-b41b-4510-93ae-36239d4374ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:22:57.393226 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76555f3-b41b-4510-93ae-36239d4374ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:22:57.394664 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76555f3-b41b-4510-93ae-36239d4374ac service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:22:57.404619 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1324/2644] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:22:57 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:23:07.399878 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=12 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:23:48.987796 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a894819-5b1f-4d41-8a5b-86909b73e55e None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:23:48.988337 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a894819-5b1f-4d41-8a5b-86909b73e55e None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:23:48.988337 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a894819-5b1f-4d41-8a5b-86909b73e55e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:23:49.008375 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-3a894819-5b1f-4d41-8a5b-86909b73e55e None None] MFA Rules not processed for user `da799b0044f54da6835aa06e43ce1a6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:23:49.009570 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-3a894819-5b1f-4d41-8a5b-86909b73e55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:23:49.016990 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1321/2645] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jul 18 17:23:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 17:23:49.057347 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76555f3-b41b-4510-93ae-36239d4374ac service neutron] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:23:49.078697 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76555f3-b41b-4510-93ae-36239d4374ac service neutron] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:23:49.083724 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081a95be-4397-4d38-ae12-880d0c6a9358 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43854d7a6df2466eb7976fabe6011767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:23:49.084370 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a95be-4397-4d38-ae12-880d0c6a9358 service swift] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:23:49.084650 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a95be-4397-4d38-ae12-880d0c6a9358 service swift] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:23:49.084925 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a95be-4397-4d38-ae12-880d0c6a9358 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:23:49.092883 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081a95be-4397-4d38-ae12-880d0c6a9358 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:23:49.095197 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081a95be-4397-4d38-ae12-880d0c6a9358 service swift] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:23:49.103110 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1325/2646] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jul 18 17:23:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 17:23:59.003102 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:23:59.099971 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=9 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:24:04.957011 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec4261-fc85-46b0-a63e-e681c443a232 None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:24:04.957672 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec4261-fc85-46b0-a63e-e681c443a232 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:24:04.958196 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec4261-fc85-46b0-a63e-e681c443a232 None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:24:04.959604 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1322/2647] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jul 18 17:24:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 17:24:04.965698 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0678d310-6a9b-44f4-88bf-ef260ef8f35f None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:24:04.966203 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0678d310-6a9b-44f4-88bf-ef260ef8f35f None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:24:04.966311 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0678d310-6a9b-44f4-88bf-ef260ef8f35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:24:04.985325 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-0678d310-6a9b-44f4-88bf-ef260ef8f35f None None] MFA Rules not processed for user `5b868bfa44b94ca5b470166d39f20e54`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:24:04.986602 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-0678d310-6a9b-44f4-88bf-ef260ef8f35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:24:05.055895 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1326/2648] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jul 18 17:24:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:24:05.068420 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ec4261-fc85-46b0-a63e-e681c443a232 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:24:05.080088 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ec4261-fc85-46b0-a63e-e681c443a232 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:24:05.083628 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865b3ded-f830-492d-98ac-1907ab61934e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0887908bcb0d4072bf6acb4cbaf1af83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:24:05.084265 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b3ded-f830-492d-98ac-1907ab61934e service glance] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:24:05.084569 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b3ded-f830-492d-98ac-1907ab61934e service glance] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:24:05.084850 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b3ded-f830-492d-98ac-1907ab61934e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:24:05.095556 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865b3ded-f830-492d-98ac-1907ab61934e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:24:05.096990 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865b3ded-f830-492d-98ac-1907ab61934e service glance] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:24:05.107533 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1323/2649] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:24:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:24:05.355829 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a118d-cacf-479f-afb9-95cba9572d45 None None] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:24:05.356121 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a118d-cacf-479f-afb9-95cba9572d45 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:24:05.356402 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a118d-cacf-479f-afb9-95cba9572d45 None None] PATH_INFO: `/` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:24:05.357831 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1327/2650] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jul 18 17:24:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 17:24:05.363296 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c01e68-f0d1-4dbc-9b44-bc53db0bfcec None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:24:05.363631 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c01e68-f0d1-4dbc-9b44-bc53db0bfcec None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:24:05.363915 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c01e68-f0d1-4dbc-9b44-bc53db0bfcec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:24:05.381854 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-40c01e68-f0d1-4dbc-9b44-bc53db0bfcec None None] MFA Rules not processed for user `7a6054dbfbbc487794d2a32061b0ac82`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:24:05.383005 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-40c01e68-f0d1-4dbc-9b44-bc53db0bfcec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:24:05.389428 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1324/2651] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jul 18 17:24:05 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:24:05.401137 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a118d-cacf-479f-afb9-95cba9572d45 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:24:05.413153 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a118d-cacf-479f-afb9-95cba9572d45 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:24:05.416737 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b265ea0d-fcf1-412f-a592-175c15e8b22c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:24:05.417377 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b265ea0d-fcf1-412f-a592-175c15e8b22c service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:24:05.417663 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b265ea0d-fcf1-412f-a592-175c15e8b22c service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:24:05.417949 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b265ea0d-fcf1-412f-a592-175c15e8b22c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:24:05.425683 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b265ea0d-fcf1-412f-a592-175c15e8b22c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:24:05.427374 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b265ea0d-fcf1-412f-a592-175c15e8b22c service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:24:05.435335 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1328/2652] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:24:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:24:05.520234 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21888cb2-81a9-43e0-9de3-185908010c6d None None] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:24:05.520525 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21888cb2-81a9-43e0-9de3-185908010c6d None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:24:05.520806 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21888cb2-81a9-43e0-9de3-185908010c6d None None] PATH_INFO: `/` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:24:05.522226 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1325/2653] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jul 18 17:24:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 17:24:05.527455 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d4d08-399f-4bb2-96b1-4be4f3642606 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:24:05.527770 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d4d08-399f-4bb2-96b1-4be4f3642606 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:24:05.528047 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d4d08-399f-4bb2-96b1-4be4f3642606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:24:05.546150 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-768d4d08-399f-4bb2-96b1-4be4f3642606 None None] MFA Rules not processed for user `5b868bfa44b94ca5b470166d39f20e54`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:24:05.548007 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-768d4d08-399f-4bb2-96b1-4be4f3642606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:24:05.557492 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1329/2654] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jul 18 17:24:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 17:24:05.570165 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21888cb2-81a9-43e0-9de3-185908010c6d None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:24:05.581217 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21888cb2-81a9-43e0-9de3-185908010c6d None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:24:05.585041 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c014e462-e5ff-454b-8954-33a97f127f0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:24:05.585688 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c014e462-e5ff-454b-8954-33a97f127f0b service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:24:05.585979 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c014e462-e5ff-454b-8954-33a97f127f0b service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:24:05.586261 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c014e462-e5ff-454b-8954-33a97f127f0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:24:05.596783 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c014e462-e5ff-454b-8954-33a97f127f0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:24:05.598217 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c014e462-e5ff-454b-8954-33a97f127f0b service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:24:05.613243 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1326/2655] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:24:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:24:07.852698 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37988dc3-e0c0-42d7-b16d-e11d2c5c47bd None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:24:07.853150 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37988dc3-e0c0-42d7-b16d-e11d2c5c47bd None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:24:07.853250 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37988dc3-e0c0-42d7-b16d-e11d2c5c47bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:24:07.871279 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-37988dc3-e0c0-42d7-b16d-e11d2c5c47bd None None] MFA Rules not processed for user `da799b0044f54da6835aa06e43ce1a6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:24:07.872513 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-37988dc3-e0c0-42d7-b16d-e11d2c5c47bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:24:07.880746 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1330/2656] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jul 18 17:24:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:24:07.892373 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c014e462-e5ff-454b-8954-33a97f127f0b service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:24:07.903785 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c014e462-e5ff-454b-8954-33a97f127f0b service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:24:07.909040 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ef410-b8fa-4565-ab00-c4d8839fae30 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43854d7a6df2466eb7976fabe6011767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:24:07.909684 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ef410-b8fa-4565-ab00-c4d8839fae30 service swift] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:24:07.909970 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ef410-b8fa-4565-ab00-c4d8839fae30 service swift] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:24:07.910248 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ef410-b8fa-4565-ab00-c4d8839fae30 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:24:07.918780 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982ef410-b8fa-4565-ab00-c4d8839fae30 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:24:07.920439 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982ef410-b8fa-4565-ab00-c4d8839fae30 service swift] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:24:07.929550 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1327/2657] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jul 18 17:24:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 17:24:17.866254 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=43 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:24:17.925946 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=21 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:24:25.372862 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7fe3ff-8aba-465f-9014-9a1f5c3a1cf8 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:24:25.373336 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7fe3ff-8aba-465f-9014-9a1f5c3a1cf8 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:24:25.373437 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7fe3ff-8aba-465f-9014-9a1f5c3a1cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:24:25.392045 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-3d7fe3ff-8aba-465f-9014-9a1f5c3a1cf8 None None] MFA Rules not processed for user `da799b0044f54da6835aa06e43ce1a6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:24:25.393230 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-3d7fe3ff-8aba-465f-9014-9a1f5c3a1cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:24:25.400340 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1331/2658] 10.222.0.26 () {58 vars in 967 bytes} [Mon Jul 18 17:24:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:24:25.412484 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ef410-b8fa-4565-ab00-c4d8839fae30 service swift] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:24:25.423685 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ef410-b8fa-4565-ab00-c4d8839fae30 service swift] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:24:25.429825 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167b1126-3d82-47c7-ac49-2b65feb964bd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43854d7a6df2466eb7976fabe6011767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:24:25.430600 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b1126-3d82-47c7-ac49-2b65feb964bd service swift] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:24:25.430892 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b1126-3d82-47c7-ac49-2b65feb964bd service swift] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:24:25.431167 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b1126-3d82-47c7-ac49-2b65feb964bd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:24:25.441038 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167b1126-3d82-47c7-ac49-2b65feb964bd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:24:25.442910 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167b1126-3d82-47c7-ac49-2b65feb964bd service swift] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:24:25.451264 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1328/2659] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jul 18 17:24:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 17:24:27.721877 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd7bcd-d031-49a8-aff6-5501455c5c47 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:24:27.722383 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd7bcd-d031-49a8-aff6-5501455c5c47 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:24:27.722488 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd7bcd-d031-49a8-aff6-5501455c5c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:24:27.740903 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-8ddd7bcd-d031-49a8-aff6-5501455c5c47 None None] MFA Rules not processed for user `da799b0044f54da6835aa06e43ce1a6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:24:27.742095 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-8ddd7bcd-d031-49a8-aff6-5501455c5c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:24:27.749734 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1332/2660] 10.222.0.26 () {58 vars in 967 bytes} [Mon Jul 18 17:24:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:24:27.762001 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167b1126-3d82-47c7-ac49-2b65feb964bd service swift] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:24:27.772653 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167b1126-3d82-47c7-ac49-2b65feb964bd service swift] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:24:27.778020 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bd4cf2-e9dd-4ffa-838e-389148593865 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43854d7a6df2466eb7976fabe6011767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:24:27.778738 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd4cf2-e9dd-4ffa-838e-389148593865 service swift] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:24:27.779027 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd4cf2-e9dd-4ffa-838e-389148593865 service swift] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:24:27.779306 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd4cf2-e9dd-4ffa-838e-389148593865 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:24:27.788278 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bd4cf2-e9dd-4ffa-838e-389148593865 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:24:27.789858 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bd4cf2-e9dd-4ffa-838e-389148593865 service swift] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:24:27.798677 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1329/2661] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jul 18 17:24:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 17:24:35.944923 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd7bcd-d031-49a8-aff6-5501455c5c47 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:24:35.946352 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-8ddd7bcd-d031-49a8-aff6-5501455c5c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:24:36.023676 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd7bcd-d031-49a8-aff6-5501455c5c47 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:24:36.028567 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75f0d1f-4c97-4e6b-b01e-655929955810 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70f704a231a5474e8067a6c2dd88b355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:24:36.029216 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f0d1f-4c97-4e6b-b01e-655929955810 service placement] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:24:36.029515 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f0d1f-4c97-4e6b-b01e-655929955810 service placement] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:24:36.029791 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f0d1f-4c97-4e6b-b01e-655929955810 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:24:36.031641 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-d75f0d1f-4c97-4e6b-b01e-655929955810 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:24:36.042807 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75f0d1f-4c97-4e6b-b01e-655929955810 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:24:36.044391 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75f0d1f-4c97-4e6b-b01e-655929955810 service placement] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:24:36.054438 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1333/2662] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:24:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:24:36.071896 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bd4cf2-e9dd-4ffa-838e-389148593865 service swift] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:24:36.148221 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bd4cf2-e9dd-4ffa-838e-389148593865 service swift] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:24:36.152851 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e0fda8-d487-48da-89ee-2cc8bf7bfc4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:24:36.153716 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e0fda8-d487-48da-89ee-2cc8bf7bfc4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:24:36.155662 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e0fda8-d487-48da-89ee-2cc8bf7bfc4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:24:36.155662 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e0fda8-d487-48da-89ee-2cc8bf7bfc4a admin admin] PATH_INFO: `/v3/users/a6ebcad94c6048c8b174cfcee63a15a7` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:24:36.155662 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e0fda8-d487-48da-89ee-2cc8bf7bfc4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6ebcad94c6048c8b174cfcee63a15a7)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:24:36.158008 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e0fda8-d487-48da-89ee-2cc8bf7bfc4a admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:24:36.306751 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-36e0fda8-d487-48da-89ee-2cc8bf7bfc4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6ebcad94c6048c8b174cfcee63a15a7'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:24:36.388259 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-36e0fda8-d487-48da-89ee-2cc8bf7bfc4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6ebcad94c6048c8b174cfcee63a15a7'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:24:36.396047 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-36e0fda8-d487-48da-89ee-2cc8bf7bfc4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6ebcad94c6048c8b174cfcee63a15a7'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:24:36.408391 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-36e0fda8-d487-48da-89ee-2cc8bf7bfc4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6ebcad94c6048c8b174cfcee63a15a7'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:24:36.409036 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-36e0fda8-d487-48da-89ee-2cc8bf7bfc4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6ebcad94c6048c8b174cfcee63a15a7'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:24:36.414965 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1330/2663] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 17:24:36 2022] DELETE /identity/v3/users/a6ebcad94c6048c8b174cfcee63a15a7 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:24:36.661496 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75f0d1f-4c97-4e6b-b01e-655929955810 service placement] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:24:36.673298 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75f0d1f-4c97-4e6b-b01e-655929955810 service placement] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:24:36.736573 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6610b973-6865-4374-9a22-e1bf1cacbf17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:24:36.737437 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610b973-6865-4374-9a22-e1bf1cacbf17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:24:36.737722 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610b973-6865-4374-9a22-e1bf1cacbf17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:24:36.738049 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610b973-6865-4374-9a22-e1bf1cacbf17 admin admin] PATH_INFO: `/v3/projects/0d565c1992f5420fa5166d2cfb6f1ab2` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:24:36.739221 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6610b973-6865-4374-9a22-e1bf1cacbf17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d565c1992f5420fa5166d2cfb6f1ab2)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:24:36.741319 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6610b973-6865-4374-9a22-e1bf1cacbf17 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:24:36.822269 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-6610b973-6865-4374-9a22-e1bf1cacbf17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d565c1992f5420fa5166d2cfb6f1ab2', 'request_id': 'req-6610b973-6865-4374-9a22-e1bf1cacbf17'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:24:36.828591 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-6610b973-6865-4374-9a22-e1bf1cacbf17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d565c1992f5420fa5166d2cfb6f1ab2', 'request_id': 'req-6610b973-6865-4374-9a22-e1bf1cacbf17'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:24:36.881647 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-6610b973-6865-4374-9a22-e1bf1cacbf17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d565c1992f5420fa5166d2cfb6f1ab2', 'request_id': 'req-6610b973-6865-4374-9a22-e1bf1cacbf17'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:24:36.892998 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-6610b973-6865-4374-9a22-e1bf1cacbf17 admin admin] The token cache is being invalidate because project 0d565c1992f5420fa5166d2cfb6f1ab2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 17:24:36.893289 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-6610b973-6865-4374-9a22-e1bf1cacbf17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:24:36.893946 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-6610b973-6865-4374-9a22-e1bf1cacbf17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:24:36.895502 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1334/2664] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 17:24:36 2022] DELETE /identity/v3/projects/0d565c1992f5420fa5166d2cfb6f1ab2 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:24:36.899484 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e0fda8-d487-48da-89ee-2cc8bf7bfc4a admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:24:36.902136 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-36e0fda8-d487-48da-89ee-2cc8bf7bfc4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:24:36.915575 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e0fda8-d487-48da-89ee-2cc8bf7bfc4a admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:24:36.960767 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:24:36.973346 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f01d19-2088-4983-bf3a-ac5733a173c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:24:36.974286 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f01d19-2088-4983-bf3a-ac5733a173c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:24:36.974593 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f01d19-2088-4983-bf3a-ac5733a173c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:24:36.974869 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f01d19-2088-4983-bf3a-ac5733a173c6 admin admin] PATH_INFO: `/v3/users/c8988216d1bc4f7186900a38724e2720` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:24:36.976099 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f01d19-2088-4983-bf3a-ac5733a173c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8988216d1bc4f7186900a38724e2720)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:24:36.978380 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f01d19-2088-4983-bf3a-ac5733a173c6 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:24:37.138748 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-21f01d19-2088-4983-bf3a-ac5733a173c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8988216d1bc4f7186900a38724e2720'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:24:37.228580 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-21f01d19-2088-4983-bf3a-ac5733a173c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8988216d1bc4f7186900a38724e2720'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:24:37.235907 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-21f01d19-2088-4983-bf3a-ac5733a173c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8988216d1bc4f7186900a38724e2720'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:24:37.247768 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-21f01d19-2088-4983-bf3a-ac5733a173c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8988216d1bc4f7186900a38724e2720'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:24:37.248417 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-21f01d19-2088-4983-bf3a-ac5733a173c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8988216d1bc4f7186900a38724e2720'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:24:37.254744 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1331/2665] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 17:24:36 2022] DELETE /identity/v3/users/c8988216d1bc4f7186900a38724e2720 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:24:37.498477 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6610b973-6865-4374-9a22-e1bf1cacbf17 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:24:37.500129 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-6610b973-6865-4374-9a22-e1bf1cacbf17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:24:37.507782 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:24:37.513149 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6610b973-6865-4374-9a22-e1bf1cacbf17 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:24:37.574059 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc90672-390b-42b6-b253-7f1c06682ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:24:37.574938 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc90672-390b-42b6-b253-7f1c06682ec0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:24:37.575225 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc90672-390b-42b6-b253-7f1c06682ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:24:37.575515 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc90672-390b-42b6-b253-7f1c06682ec0 admin admin] PATH_INFO: `/v3/projects/ec70c8db318d4f37b21bea763ebd39cc` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:24:37.576735 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc90672-390b-42b6-b253-7f1c06682ec0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec70c8db318d4f37b21bea763ebd39cc)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:24:37.578774 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc90672-390b-42b6-b253-7f1c06682ec0 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:24:37.669156 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-8dc90672-390b-42b6-b253-7f1c06682ec0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec70c8db318d4f37b21bea763ebd39cc', 'request_id': 'req-8dc90672-390b-42b6-b253-7f1c06682ec0'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:24:37.675546 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-8dc90672-390b-42b6-b253-7f1c06682ec0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec70c8db318d4f37b21bea763ebd39cc', 'request_id': 'req-8dc90672-390b-42b6-b253-7f1c06682ec0'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:24:37.754054 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-8dc90672-390b-42b6-b253-7f1c06682ec0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec70c8db318d4f37b21bea763ebd39cc', 'request_id': 'req-8dc90672-390b-42b6-b253-7f1c06682ec0'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:24:37.764981 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-8dc90672-390b-42b6-b253-7f1c06682ec0 admin admin] The token cache is being invalidate because project ec70c8db318d4f37b21bea763ebd39cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 17:24:37.765267 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-8dc90672-390b-42b6-b253-7f1c06682ec0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:24:37.766449 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-8dc90672-390b-42b6-b253-7f1c06682ec0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:24:37.768039 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1335/2666] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 17:24:37 2022] DELETE /identity/v3/projects/ec70c8db318d4f37b21bea763ebd39cc => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:24:37.772995 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce3d04d-d2df-4a2d-a1c5-28fd563de800 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:24:37.773316 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce3d04d-d2df-4a2d-a1c5-28fd563de800 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:24:37.773597 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce3d04d-d2df-4a2d-a1c5-28fd563de800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:24:37.791294 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-7ce3d04d-d2df-4a2d-a1c5-28fd563de800 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:24:37.792537 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-7ce3d04d-d2df-4a2d-a1c5-28fd563de800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:24:37.974293 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1332/2667] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:24:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:24:37.982186 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce819e-b069-409a-bf23-263ae428f656 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:24:37.982479 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce819e-b069-409a-bf23-263ae428f656 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:24:37.982760 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce819e-b069-409a-bf23-263ae428f656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:24:38.003390 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-71ce819e-b069-409a-bf23-263ae428f656 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:24:38.004688 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-71ce819e-b069-409a-bf23-263ae428f656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:24:38.012574 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1336/2668] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:24:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:24:38.016850 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce3d04d-d2df-4a2d-a1c5-28fd563de800 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:24:38.026528 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce3d04d-d2df-4a2d-a1c5-28fd563de800 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:24:38.031060 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44454215-6afb-4d1a-ada3-879c5b840a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:24:38.031749 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44454215-6afb-4d1a-ada3-879c5b840a77 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:24:38.032038 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44454215-6afb-4d1a-ada3-879c5b840a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:24:38.032340 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44454215-6afb-4d1a-ada3-879c5b840a77 admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:24:38.033086 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44454215-6afb-4d1a-ada3-879c5b840a77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 17:24:38.033396 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44454215-6afb-4d1a-ada3-879c5b840a77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:24:38.034983 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44454215-6afb-4d1a-ada3-879c5b840a77 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:24:38.046476 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1333/2669] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 17:24:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 17:24:38.051117 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ce819e-b069-409a-bf23-263ae428f656 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:24:38.060549 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ce819e-b069-409a-bf23-263ae428f656 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:24:38.065285 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53efbc71-1945-4d7a-859d-6c0755ad49b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:24:38.065935 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53efbc71-1945-4d7a-859d-6c0755ad49b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:24:38.066222 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53efbc71-1945-4d7a-859d-6c0755ad49b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:24:38.066501 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53efbc71-1945-4d7a-859d-6c0755ad49b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:24:38.067257 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53efbc71-1945-4d7a-859d-6c0755ad49b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:24:38.069197 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53efbc71-1945-4d7a-859d-6c0755ad49b3 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:24:38.157245 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1337/2670] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 17:24:38 2022] POST /identity/v3/projects => generated 379 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 17:24:38.161382 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44454215-6afb-4d1a-ada3-879c5b840a77 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:24:38.171061 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44454215-6afb-4d1a-ada3-879c5b840a77 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:24:38.221246 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:24:38.228115 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c64617-16e7-4cc6-a246-d92ba579f2e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:24:38.228816 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c64617-16e7-4cc6-a246-d92ba579f2e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:24:38.228937 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c64617-16e7-4cc6-a246-d92ba579f2e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:24:38.229189 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c64617-16e7-4cc6-a246-d92ba579f2e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:24:38.229873 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c64617-16e7-4cc6-a246-d92ba579f2e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:24:38.231798 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c64617-16e7-4cc6-a246-d92ba579f2e3 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:24:38.412808 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1334/2671] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 17:24:38 2022] POST /identity/v3/users => generated 290 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:24:38.416758 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53efbc71-1945-4d7a-859d-6c0755ad49b3 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:24:38.427721 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53efbc71-1945-4d7a-859d-6c0755ad49b3 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:24:38.432257 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9aaf9b-9da7-457d-bec8-5d6fce0ed2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:24:38.432929 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9aaf9b-9da7-457d-bec8-5d6fce0ed2e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:24:38.433211 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9aaf9b-9da7-457d-bec8-5d6fce0ed2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:24:38.433484 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9aaf9b-9da7-457d-bec8-5d6fce0ed2e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:24:38.434203 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9aaf9b-9da7-457d-bec8-5d6fce0ed2e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:24:38.435819 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9aaf9b-9da7-457d-bec8-5d6fce0ed2e8 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:24:38.446832 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1338/2672] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 17:24:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:24:38.450776 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c64617-16e7-4cc6-a246-d92ba579f2e3 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:24:38.463246 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c64617-16e7-4cc6-a246-d92ba579f2e3 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:24:38.467764 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4093d87b-f663-45d9-80e5-d6e00cdcc472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:24:38.468431 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093d87b-f663-45d9-80e5-d6e00cdcc472 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:24:38.468749 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093d87b-f663-45d9-80e5-d6e00cdcc472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:24:38.469310 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093d87b-f663-45d9-80e5-d6e00cdcc472 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:24:38.469816 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4093d87b-f663-45d9-80e5-d6e00cdcc472 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:24:38.471390 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4093d87b-f663-45d9-80e5-d6e00cdcc472 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:24:38.481885 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1335/2673] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 17:24:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:24:38.485950 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9aaf9b-9da7-457d-bec8-5d6fce0ed2e8 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:24:38.496106 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9aaf9b-9da7-457d-bec8-5d6fce0ed2e8 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:24:38.501249 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c972ec0-ebf0-49d9-8c6c-5572d79a90cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:24:38.502128 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c972ec0-ebf0-49d9-8c6c-5572d79a90cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:24:38.502411 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c972ec0-ebf0-49d9-8c6c-5572d79a90cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:24:38.502687 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c972ec0-ebf0-49d9-8c6c-5572d79a90cd admin admin] PATH_INFO: `/v3/projects/e4afeb76581942e4a0d72e4568907201/users/e1fc02859caf4ac09a85a4e5f962c14b/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:24:38.517595 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c972ec0-ebf0-49d9-8c6c-5572d79a90cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4afeb76581942e4a0d72e4568907201, user_id=e1fc02859caf4ac09a85a4e5f962c14b, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:24:38.520301 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c972ec0-ebf0-49d9-8c6c-5572d79a90cd admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:24:38.612276 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1339/2674] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 17:24:38 2022] PUT /identity/v3/projects/e4afeb76581942e4a0d72e4568907201/users/e1fc02859caf4ac09a85a4e5f962c14b/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:24:38.617049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f53b2b-3da0-458b-96dc-6f8f07361420 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:24:38.617398 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f53b2b-3da0-458b-96dc-6f8f07361420 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:24:38.617686 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f53b2b-3da0-458b-96dc-6f8f07361420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:24:38.636230 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-44f53b2b-3da0-458b-96dc-6f8f07361420 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:24:38.637422 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-44f53b2b-3da0-458b-96dc-6f8f07361420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:24:38.696506 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1336/2675] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:24:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:24:38.706289 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c972ec0-ebf0-49d9-8c6c-5572d79a90cd admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:24:38.708043 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-7c972ec0-ebf0-49d9-8c6c-5572d79a90cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:24:38.743613 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:24:38.769521 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c972ec0-ebf0-49d9-8c6c-5572d79a90cd admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:24:38.773283 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b85b20-22bf-4cd5-92c1-923f07a85b19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:24:38.773937 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b85b20-22bf-4cd5-92c1-923f07a85b19 service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:24:38.781630 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b85b20-22bf-4cd5-92c1-923f07a85b19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:24:38.781630 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b85b20-22bf-4cd5-92c1-923f07a85b19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:24:38.784398 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b85b20-22bf-4cd5-92c1-923f07a85b19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:24:38.785834 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b85b20-22bf-4cd5-92c1-923f07a85b19 service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:24:38.795249 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1340/2676] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:24:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:24:44.189143 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a9d47-9780-433a-a4a1-5d3e5f1cf518 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:24:44.189620 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a9d47-9780-433a-a4a1-5d3e5f1cf518 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:24:44.189738 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a9d47-9780-433a-a4a1-5d3e5f1cf518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:24:44.227320 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-bd2a9d47-9780-433a-a4a1-5d3e5f1cf518 None None] MFA Rules not processed for user `e1fc02859caf4ac09a85a4e5f962c14b`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:24:44.228594 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-bd2a9d47-9780-433a-a4a1-5d3e5f1cf518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:24:44.296103 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1337/2677] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:24:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:24:44.300953 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b85b20-22bf-4cd5-92c1-923f07a85b19 service neutron] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:24:44.310888 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b85b20-22bf-4cd5-92c1-923f07a85b19 service neutron] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:24:44.316272 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7621f2-b0ba-4e95-92a0-59c19051f0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:24:44.316958 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7621f2-b0ba-4e95-92a0-59c19051f0dc admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:24:44.317245 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7621f2-b0ba-4e95-92a0-59c19051f0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:24:44.317524 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7621f2-b0ba-4e95-92a0-59c19051f0dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:24:44.318386 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7621f2-b0ba-4e95-92a0-59c19051f0dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:24:44.320445 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7621f2-b0ba-4e95-92a0-59c19051f0dc admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:24:44.419956 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1341/2678] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 17:24:44 2022] POST /identity/v3/projects => generated 379 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:24:44.423952 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2a9d47-9780-433a-a4a1-5d3e5f1cf518 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:24:44.434764 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2a9d47-9780-433a-a4a1-5d3e5f1cf518 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:24:44.487123 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:24:44.493933 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d95235-447b-4d07-a01c-d09e00c84731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:24:44.498412 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d95235-447b-4d07-a01c-d09e00c84731 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:24:44.498412 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d95235-447b-4d07-a01c-d09e00c84731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:24:44.498412 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d95235-447b-4d07-a01c-d09e00c84731 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:24:44.498412 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d95235-447b-4d07-a01c-d09e00c84731 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:24:44.498412 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d95235-447b-4d07-a01c-d09e00c84731 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:24:44.586727 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1338/2679] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 17:24:44 2022] POST /identity/v3/users => generated 296 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:24:44.588047 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7621f2-b0ba-4e95-92a0-59c19051f0dc admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:24:44.604096 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7621f2-b0ba-4e95-92a0-59c19051f0dc admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:24:44.608854 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a56a92f-6a66-438b-b54c-7a8b49debd9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:24:44.610028 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a56a92f-6a66-438b-b54c-7a8b49debd9c admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:24:44.610317 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a56a92f-6a66-438b-b54c-7a8b49debd9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:24:44.610612 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a56a92f-6a66-438b-b54c-7a8b49debd9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:24:44.611336 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a56a92f-6a66-438b-b54c-7a8b49debd9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:24:44.613019 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a56a92f-6a66-438b-b54c-7a8b49debd9c admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:24:44.624333 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1342/2680] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 17:24:44 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:24:44.628910 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d95235-447b-4d07-a01c-d09e00c84731 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:24:44.639288 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d95235-447b-4d07-a01c-d09e00c84731 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:24:44.644831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d2cddc-93ea-4723-811f-d947e5828de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:24:44.645898 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d2cddc-93ea-4723-811f-d947e5828de5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:24:44.646186 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d2cddc-93ea-4723-811f-d947e5828de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:24:44.646462 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d2cddc-93ea-4723-811f-d947e5828de5 admin admin] PATH_INFO: `/v3/projects/ae03122aa4c944249656119878e22d25/users/0f52c81946ee49bb805781b3e69e4bbb/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:24:44.662940 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d2cddc-93ea-4723-811f-d947e5828de5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae03122aa4c944249656119878e22d25, user_id=0f52c81946ee49bb805781b3e69e4bbb, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:24:44.666008 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d2cddc-93ea-4723-811f-d947e5828de5 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:24:44.773992 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1339/2681] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 17:24:44 2022] PUT /identity/v3/projects/ae03122aa4c944249656119878e22d25/users/0f52c81946ee49bb805781b3e69e4bbb/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 17:24:44.778093 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a56a92f-6a66-438b-b54c-7a8b49debd9c admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:24:44.788595 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a56a92f-6a66-438b-b54c-7a8b49debd9c admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:24:44.853326 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3659208-521a-4dce-bce8-90e574578475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:24:44.853804 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3659208-521a-4dce-bce8-90e574578475 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:24:44.854088 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3659208-521a-4dce-bce8-90e574578475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:24:44.854366 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3659208-521a-4dce-bce8-90e574578475 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:24:44.855097 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3659208-521a-4dce-bce8-90e574578475 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:24:44.856737 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3659208-521a-4dce-bce8-90e574578475 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:24:44.869288 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1343/2682] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 17:24:44 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:24:44.873390 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d2cddc-93ea-4723-811f-d947e5828de5 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:24:44.885629 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d2cddc-93ea-4723-811f-d947e5828de5 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:24:44.891320 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdd24dc-702a-4152-8dd5-56b773d92ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:24:44.892280 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd24dc-702a-4152-8dd5-56b773d92ced admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:24:44.892560 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd24dc-702a-4152-8dd5-56b773d92ced admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:24:44.892832 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd24dc-702a-4152-8dd5-56b773d92ced admin admin] PATH_INFO: `/v3/projects/ae03122aa4c944249656119878e22d25/users/0f52c81946ee49bb805781b3e69e4bbb/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:24:44.895179 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdd24dc-702a-4152-8dd5-56b773d92ced admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae03122aa4c944249656119878e22d25, user_id=0f52c81946ee49bb805781b3e69e4bbb, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:24:44.897855 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdd24dc-702a-4152-8dd5-56b773d92ced admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:24:44.909833 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1340/2683] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 17:24:44 2022] PUT /identity/v3/projects/ae03122aa4c944249656119878e22d25/users/0f52c81946ee49bb805781b3e69e4bbb/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:24:50.453538 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c408f-1039-4d39-bb33-8f717e8396dc None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:24:50.454150 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c408f-1039-4d39-bb33-8f717e8396dc None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:24:50.454265 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c408f-1039-4d39-bb33-8f717e8396dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:24:50.483189 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-201c408f-1039-4d39-bb33-8f717e8396dc None None] MFA Rules not processed for user `0f52c81946ee49bb805781b3e69e4bbb`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:24:50.485241 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-201c408f-1039-4d39-bb33-8f717e8396dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:24:50.580795 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1344/2684] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:24:50 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:24:50.595163 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdd24dc-702a-4152-8dd5-56b773d92ced admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:24:50.601789 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-8fdd24dc-702a-4152-8dd5-56b773d92ced admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:24:50.706172 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdd24dc-702a-4152-8dd5-56b773d92ced admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:24:50.711986 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64b4258-0bef-4f9b-90d3-c1f574c04067 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:24:50.712656 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b4258-0bef-4f9b-90d3-c1f574c04067 service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:24:50.712944 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b4258-0bef-4f9b-90d3-c1f574c04067 service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:24:50.713226 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b4258-0bef-4f9b-90d3-c1f574c04067 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:24:50.788912 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64b4258-0bef-4f9b-90d3-c1f574c04067 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:24:50.788912 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64b4258-0bef-4f9b-90d3-c1f574c04067 service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:24:50.797378 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1341/2685] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:24:50 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:24:59.160748 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201c408f-1039-4d39-bb33-8f717e8396dc None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:24:59.162231 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-201c408f-1039-4d39-bb33-8f717e8396dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:24:59.185841 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:24:59.223416 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201c408f-1039-4d39-bb33-8f717e8396dc None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:24:59.227143 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6783c8ff-fc5c-484e-864a-dbb2eef2dda6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:24:59.227834 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6783c8ff-fc5c-484e-864a-dbb2eef2dda6 service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:24:59.228116 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6783c8ff-fc5c-484e-864a-dbb2eef2dda6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:24:59.228390 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6783c8ff-fc5c-484e-864a-dbb2eef2dda6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:24:59.230196 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-6783c8ff-fc5c-484e-864a-dbb2eef2dda6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:24:59.314818 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6783c8ff-fc5c-484e-864a-dbb2eef2dda6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:24:59.316288 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6783c8ff-fc5c-484e-864a-dbb2eef2dda6 service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:24:59.338060 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1345/2686] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:24:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:25:00.792464 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=82,keystone:INSERT=5 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:25:06.920457 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64b4258-0bef-4f9b-90d3-c1f574c04067 service nova] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:25:06.974858 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64b4258-0bef-4f9b-90d3-c1f574c04067 service nova] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:25:06.978448 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531ed1e9-bf7a-46ff-9da6-e9114b3623f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:25:06.979084 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ed1e9-bf7a-46ff-9da6-e9114b3623f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:25:06.979362 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ed1e9-bf7a-46ff-9da6-e9114b3623f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:25:06.979676 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ed1e9-bf7a-46ff-9da6-e9114b3623f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:25:06.981402 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-531ed1e9-bf7a-46ff-9da6-e9114b3623f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:25:07.042058 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531ed1e9-bf7a-46ff-9da6-e9114b3623f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:25:07.043476 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531ed1e9-bf7a-46ff-9da6-e9114b3623f4 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:25:07.053024 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1342/2687] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:25:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:25:08.949276 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6488ce3b-5f99-4cc8-984c-68d441628465 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:25:08.949802 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6488ce3b-5f99-4cc8-984c-68d441628465 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:25:08.949802 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6488ce3b-5f99-4cc8-984c-68d441628465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:25:08.975914 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-6488ce3b-5f99-4cc8-984c-68d441628465 None None] MFA Rules not processed for user `e1fc02859caf4ac09a85a4e5f962c14b`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:25:08.977534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-6488ce3b-5f99-4cc8-984c-68d441628465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:25:08.985229 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1346/2688] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:25:08 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:25:10.497556 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531ed1e9-bf7a-46ff-9da6-e9114b3623f4 service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:25:10.498950 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-531ed1e9-bf7a-46ff-9da6-e9114b3623f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:25:10.515543 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531ed1e9-bf7a-46ff-9da6-e9114b3623f4 service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:25:10.515894 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a56d6-301f-4f4e-a6ae-46fd74806776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:25:10.516752 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a56d6-301f-4f4e-a6ae-46fd74806776 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:25:10.517030 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a56d6-301f-4f4e-a6ae-46fd74806776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:25:10.517308 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a56d6-301f-4f4e-a6ae-46fd74806776 admin admin] PATH_INFO: `/v3/users/9aec7436e3a549018fd7892969870019` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:25:10.519280 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a56d6-301f-4f4e-a6ae-46fd74806776 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9aec7436e3a549018fd7892969870019)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:25:10.521321 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a56d6-301f-4f4e-a6ae-46fd74806776 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:25:10.718168 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-df8a56d6-301f-4f4e-a6ae-46fd74806776 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9aec7436e3a549018fd7892969870019'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:25:10.784641 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-df8a56d6-301f-4f4e-a6ae-46fd74806776 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9aec7436e3a549018fd7892969870019'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:25:10.792175 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-df8a56d6-301f-4f4e-a6ae-46fd74806776 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9aec7436e3a549018fd7892969870019'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:25:10.804792 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-df8a56d6-301f-4f4e-a6ae-46fd74806776 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9aec7436e3a549018fd7892969870019'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:25:10.805486 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-df8a56d6-301f-4f4e-a6ae-46fd74806776 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9aec7436e3a549018fd7892969870019'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:25:10.811427 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1343/2689] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 17:25:10 2022] DELETE /identity/v3/users/9aec7436e3a549018fd7892969870019 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:25:11.334679 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6488ce3b-5f99-4cc8-984c-68d441628465 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:25:11.353683 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6488ce3b-5f99-4cc8-984c-68d441628465 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:25:11.406361 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23ed318-770b-44c6-beb9-dfa63ecdb923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:25:11.407235 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ed318-770b-44c6-beb9-dfa63ecdb923 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:25:11.407515 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ed318-770b-44c6-beb9-dfa63ecdb923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:25:11.407806 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ed318-770b-44c6-beb9-dfa63ecdb923 admin admin] PATH_INFO: `/v3/projects/1bad1610198d4b8aacfafc8c92dc7e6d` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:25:11.409250 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23ed318-770b-44c6-beb9-dfa63ecdb923 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bad1610198d4b8aacfafc8c92dc7e6d)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:25:11.411231 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23ed318-770b-44c6-beb9-dfa63ecdb923 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:25:11.531983 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:25:11.538358 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-b23ed318-770b-44c6-beb9-dfa63ecdb923 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bad1610198d4b8aacfafc8c92dc7e6d', 'request_id': 'req-b23ed318-770b-44c6-beb9-dfa63ecdb923'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:25:11.544521 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-b23ed318-770b-44c6-beb9-dfa63ecdb923 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bad1610198d4b8aacfafc8c92dc7e6d', 'request_id': 'req-b23ed318-770b-44c6-beb9-dfa63ecdb923'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:25:11.629316 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-b23ed318-770b-44c6-beb9-dfa63ecdb923 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bad1610198d4b8aacfafc8c92dc7e6d', 'request_id': 'req-b23ed318-770b-44c6-beb9-dfa63ecdb923'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:25:11.640821 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-b23ed318-770b-44c6-beb9-dfa63ecdb923 admin admin] The token cache is being invalidate because project 1bad1610198d4b8aacfafc8c92dc7e6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 17:25:11.641106 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-b23ed318-770b-44c6-beb9-dfa63ecdb923 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:25:11.641744 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-b23ed318-770b-44c6-beb9-dfa63ecdb923 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:25:11.643276 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1347/2690] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 17:25:11 2022] DELETE /identity/v3/projects/1bad1610198d4b8aacfafc8c92dc7e6d => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:25:11.648563 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09916db-fadf-438c-ada4-142f0a52942d None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:25:11.649053 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09916db-fadf-438c-ada4-142f0a52942d None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:25:11.649174 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09916db-fadf-438c-ada4-142f0a52942d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:25:11.678266 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:25:11.698105 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-b09916db-fadf-438c-ada4-142f0a52942d None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:25:11.699314 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-b09916db-fadf-438c-ada4-142f0a52942d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:25:11.756436 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1344/2691] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:25:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:25:11.764909 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b2a926-8e82-412f-8fa5-6c837f29b2f7 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:25:11.765296 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b2a926-8e82-412f-8fa5-6c837f29b2f7 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:25:11.765666 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b2a926-8e82-412f-8fa5-6c837f29b2f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:25:11.785769 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-07b2a926-8e82-412f-8fa5-6c837f29b2f7 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:25:11.786999 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-07b2a926-8e82-412f-8fa5-6c837f29b2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:25:11.795356 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1348/2692] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:25:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:25:11.800347 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09916db-fadf-438c-ada4-142f0a52942d None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:25:11.810168 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09916db-fadf-438c-ada4-142f0a52942d None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:25:11.814831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45939dd8-f876-40de-828b-ae8cc645fc4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:25:11.815516 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45939dd8-f876-40de-828b-ae8cc645fc4e admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:25:11.815827 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45939dd8-f876-40de-828b-ae8cc645fc4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:25:11.816102 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45939dd8-f876-40de-828b-ae8cc645fc4e admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:25:11.816817 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45939dd8-f876-40de-828b-ae8cc645fc4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 17:25:11.817119 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45939dd8-f876-40de-828b-ae8cc645fc4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:25:11.818676 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45939dd8-f876-40de-828b-ae8cc645fc4e admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:25:11.830761 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1345/2693] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 17:25:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 17:25:11.835677 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b2a926-8e82-412f-8fa5-6c837f29b2f7 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:25:11.846108 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b2a926-8e82-412f-8fa5-6c837f29b2f7 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:25:11.850884 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa69b2b4-5684-40a9-a78b-d0d02eb251ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:25:11.851585 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa69b2b4-5684-40a9-a78b-d0d02eb251ca admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:25:11.851874 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa69b2b4-5684-40a9-a78b-d0d02eb251ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:25:11.852156 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa69b2b4-5684-40a9-a78b-d0d02eb251ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:25:11.852854 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa69b2b4-5684-40a9-a78b-d0d02eb251ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:25:11.854749 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa69b2b4-5684-40a9-a78b-d0d02eb251ca admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:25:11.950438 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1349/2694] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 17:25:11 2022] POST /identity/v3/projects => generated 383 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:25:11.954545 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45939dd8-f876-40de-828b-ae8cc645fc4e admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:25:11.964838 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45939dd8-f876-40de-828b-ae8cc645fc4e admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:25:12.024065 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3f6a84-e409-4132-95ee-e4ac459f8203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:25:12.024645 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f6a84-e409-4132-95ee-e4ac459f8203 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:25:12.024924 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f6a84-e409-4132-95ee-e4ac459f8203 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:25:12.025194 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f6a84-e409-4132-95ee-e4ac459f8203 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:25:12.025867 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3f6a84-e409-4132-95ee-e4ac459f8203 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:25:12.027819 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3f6a84-e409-4132-95ee-e4ac459f8203 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:25:12.132147 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1346/2695] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 17:25:11 2022] POST /identity/v3/users => generated 292 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:25:12.136341 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa69b2b4-5684-40a9-a78b-d0d02eb251ca admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:25:12.147204 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa69b2b4-5684-40a9-a78b-d0d02eb251ca admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:25:12.151837 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb553e8-b8c0-478b-a990-ee55bc831e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:25:12.152503 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb553e8-b8c0-478b-a990-ee55bc831e3e admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:25:12.152790 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb553e8-b8c0-478b-a990-ee55bc831e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:25:12.153073 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb553e8-b8c0-478b-a990-ee55bc831e3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:25:12.153807 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb553e8-b8c0-478b-a990-ee55bc831e3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:25:12.155445 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb553e8-b8c0-478b-a990-ee55bc831e3e admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:25:12.165945 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1350/2696] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 17:25:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:25:12.170069 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3f6a84-e409-4132-95ee-e4ac459f8203 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:25:12.180892 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3f6a84-e409-4132-95ee-e4ac459f8203 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:25:12.185556 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c6b05-1423-41cb-b1da-143952568bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:25:12.186215 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c6b05-1423-41cb-b1da-143952568bce admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:25:12.186518 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c6b05-1423-41cb-b1da-143952568bce admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:25:12.186783 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c6b05-1423-41cb-b1da-143952568bce admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:25:12.187540 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772c6b05-1423-41cb-b1da-143952568bce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:25:12.189251 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772c6b05-1423-41cb-b1da-143952568bce admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:25:12.200130 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1347/2697] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 17:25:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:25:12.207540 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb553e8-b8c0-478b-a990-ee55bc831e3e admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:25:12.218197 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb553e8-b8c0-478b-a990-ee55bc831e3e admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:25:12.223956 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70590d64-fdba-4a82-bd96-63b8ea7d6a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:25:12.224847 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70590d64-fdba-4a82-bd96-63b8ea7d6a90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:25:12.225127 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70590d64-fdba-4a82-bd96-63b8ea7d6a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:25:12.225424 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70590d64-fdba-4a82-bd96-63b8ea7d6a90 admin admin] PATH_INFO: `/v3/projects/60611329802847dd9da7d4d230ab32a4/users/a708e1f37fd5453198e32ec6427afcaa/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:25:12.242025 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70590d64-fdba-4a82-bd96-63b8ea7d6a90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60611329802847dd9da7d4d230ab32a4, user_id=a708e1f37fd5453198e32ec6427afcaa, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:25:12.244700 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70590d64-fdba-4a82-bd96-63b8ea7d6a90 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:25:12.344841 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1351/2698] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 17:25:12 2022] PUT /identity/v3/projects/60611329802847dd9da7d4d230ab32a4/users/a708e1f37fd5453198e32ec6427afcaa/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 17:25:12.350597 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8398dba6-a82c-4bfa-9b13-9cd740657ab8 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:25:12.350886 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8398dba6-a82c-4bfa-9b13-9cd740657ab8 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:25:12.351163 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8398dba6-a82c-4bfa-9b13-9cd740657ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:25:12.369559 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-8398dba6-a82c-4bfa-9b13-9cd740657ab8 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:25:12.370705 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-8398dba6-a82c-4bfa-9b13-9cd740657ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:25:12.437633 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1348/2699] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:25:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:25:12.447736 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70590d64-fdba-4a82-bd96-63b8ea7d6a90 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:25:12.449488 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-70590d64-fdba-4a82-bd96-63b8ea7d6a90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:25:12.526287 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70590d64-fdba-4a82-bd96-63b8ea7d6a90 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:25:12.530848 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a76d12-a6e9-4b3e-9d9d-a3588f4c9cc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:25:12.531593 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a76d12-a6e9-4b3e-9d9d-a3588f4c9cc8 service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:25:12.531883 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a76d12-a6e9-4b3e-9d9d-a3588f4c9cc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:25:12.532306 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a76d12-a6e9-4b3e-9d9d-a3588f4c9cc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:25:12.545183 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a76d12-a6e9-4b3e-9d9d-a3588f4c9cc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:25:12.546622 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a76d12-a6e9-4b3e-9d9d-a3588f4c9cc8 service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:25:12.568474 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1352/2700] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:25:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:25:17.924686 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f6525-bb7b-49e1-9010-35c6448324c8 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:25:17.925209 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f6525-bb7b-49e1-9010-35c6448324c8 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:25:17.925337 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f6525-bb7b-49e1-9010-35c6448324c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:25:17.932885 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:25:17.953820 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-399f6525-bb7b-49e1-9010-35c6448324c8 None None] MFA Rules not processed for user `a708e1f37fd5453198e32ec6427afcaa`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:25:17.956789 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-399f6525-bb7b-49e1-9010-35c6448324c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:25:18.044797 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1349/2701] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:25:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:25:18.060047 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a76d12-a6e9-4b3e-9d9d-a3588f4c9cc8 service neutron] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:25:18.061574 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-f4a76d12-a6e9-4b3e-9d9d-a3588f4c9cc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:25:18.128210 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:25:18.145302 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a76d12-a6e9-4b3e-9d9d-a3588f4c9cc8 service neutron] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:25:18.151082 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636818d6-acdb-4b06-8a89-cc5c9e2c5cb9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:25:18.151785 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636818d6-acdb-4b06-8a89-cc5c9e2c5cb9 service nova] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:25:18.152078 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636818d6-acdb-4b06-8a89-cc5c9e2c5cb9 service nova] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:25:18.152357 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636818d6-acdb-4b06-8a89-cc5c9e2c5cb9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:25:18.161363 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636818d6-acdb-4b06-8a89-cc5c9e2c5cb9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:25:18.162984 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636818d6-acdb-4b06-8a89-cc5c9e2c5cb9 service nova] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:25:18.172897 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1353/2702] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:25:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:25:21.180567 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2009a-f689-41de-b337-47ed594ea0e1 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:25:21.180567 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2009a-f689-41de-b337-47ed594ea0e1 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:25:21.180567 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2009a-f689-41de-b337-47ed594ea0e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:25:21.196541 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-57a2009a-f689-41de-b337-47ed594ea0e1 None None] MFA Rules not processed for user `a708e1f37fd5453198e32ec6427afcaa`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:25:21.197771 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-57a2009a-f689-41de-b337-47ed594ea0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:25:21.205627 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1350/2703] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:25:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:25:23.295862 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636818d6-acdb-4b06-8a89-cc5c9e2c5cb9 service nova] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:25:23.297429 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-636818d6-acdb-4b06-8a89-cc5c9e2c5cb9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:25:23.408543 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636818d6-acdb-4b06-8a89-cc5c9e2c5cb9 service nova] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:25:23.415304 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27db18d2-541b-4cd6-9c6a-a0867042bbcb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70f704a231a5474e8067a6c2dd88b355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:25:23.416327 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db18d2-541b-4cd6-9c6a-a0867042bbcb service placement] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:25:23.416823 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db18d2-541b-4cd6-9c6a-a0867042bbcb service placement] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:25:23.417116 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db18d2-541b-4cd6-9c6a-a0867042bbcb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:25:23.419256 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-27db18d2-541b-4cd6-9c6a-a0867042bbcb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:25:23.438218 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27db18d2-541b-4cd6-9c6a-a0867042bbcb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:25:23.439877 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27db18d2-541b-4cd6-9c6a-a0867042bbcb service placement] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:25:23.450915 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1354/2704] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:25:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:25:27.464642 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a2009a-f689-41de-b337-47ed594ea0e1 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:25:27.474338 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a2009a-f689-41de-b337-47ed594ea0e1 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:25:27.478029 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcb105e-c49f-4d3a-b0c6-05e68f025c18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:25:27.478612 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb105e-c49f-4d3a-b0c6-05e68f025c18 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:25:27.478897 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb105e-c49f-4d3a-b0c6-05e68f025c18 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:25:27.479172 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb105e-c49f-4d3a-b0c6-05e68f025c18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:25:27.481212 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-dbcb105e-c49f-4d3a-b0c6-05e68f025c18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:25:27.547631 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcb105e-c49f-4d3a-b0c6-05e68f025c18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:25:27.549237 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcb105e-c49f-4d3a-b0c6-05e68f025c18 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:25:27.560613 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1351/2705] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:25:27 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 17:25:32.638376 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27db18d2-541b-4cd6-9c6a-a0867042bbcb service placement] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:25:32.639829 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-27db18d2-541b-4cd6-9c6a-a0867042bbcb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:25:32.651431 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27db18d2-541b-4cd6-9c6a-a0867042bbcb service placement] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:25:32.655012 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c14bd11-12ca-4ceb-82ec-373fe2b47639 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:25:32.655690 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14bd11-12ca-4ceb-82ec-373fe2b47639 service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:25:32.655977 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14bd11-12ca-4ceb-82ec-373fe2b47639 service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:25:32.656254 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14bd11-12ca-4ceb-82ec-373fe2b47639 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:25:32.658049 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-7c14bd11-12ca-4ceb-82ec-373fe2b47639 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:25:32.673167 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c14bd11-12ca-4ceb-82ec-373fe2b47639 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:25:32.675318 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c14bd11-12ca-4ceb-82ec-373fe2b47639 service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:25:32.686803 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1355/2706] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:25:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:25:37.556062 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=63 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:25:42.681927 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=54 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:26:24.545624 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95232c4e-3b45-45bf-821a-097b6b1efbc9 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:26:24.546148 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95232c4e-3b45-45bf-821a-097b6b1efbc9 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:26:24.546148 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95232c4e-3b45-45bf-821a-097b6b1efbc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:26:24.563291 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-95232c4e-3b45-45bf-821a-097b6b1efbc9 None None] MFA Rules not processed for user `a708e1f37fd5453198e32ec6427afcaa`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:26:24.564556 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-95232c4e-3b45-45bf-821a-097b6b1efbc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:26:24.571483 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1352/2707] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:26:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:26:24.875289 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c14bd11-12ca-4ceb-82ec-373fe2b47639 service neutron] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:26:24.884272 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c14bd11-12ca-4ceb-82ec-373fe2b47639 service neutron] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:26:24.887885 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa956761-2258-481a-9906-7cb282c7ede2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:26:24.888520 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa956761-2258-481a-9906-7cb282c7ede2 service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:26:24.888827 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa956761-2258-481a-9906-7cb282c7ede2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:26:24.889099 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa956761-2258-481a-9906-7cb282c7ede2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:26:24.890835 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-aa956761-2258-481a-9906-7cb282c7ede2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:26:24.901060 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa956761-2258-481a-9906-7cb282c7ede2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:26:24.904033 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa956761-2258-481a-9906-7cb282c7ede2 service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:26:24.910458 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1356/2708] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:26:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:26:25.296179 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95232c4e-3b45-45bf-821a-097b6b1efbc9 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:26:25.309146 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95232c4e-3b45-45bf-821a-097b6b1efbc9 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:26:25.313962 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55710084-c460-41ed-b65d-f5c13eab7eee service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70f704a231a5474e8067a6c2dd88b355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:26:25.314603 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55710084-c460-41ed-b65d-f5c13eab7eee service placement] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:26:25.314881 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55710084-c460-41ed-b65d-f5c13eab7eee service placement] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:26:25.315153 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55710084-c460-41ed-b65d-f5c13eab7eee service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:26:25.317650 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-55710084-c460-41ed-b65d-f5c13eab7eee service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:26:25.331132 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55710084-c460-41ed-b65d-f5c13eab7eee service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:26:25.333060 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55710084-c460-41ed-b65d-f5c13eab7eee service placement] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:26:25.348297 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1353/2709] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:26:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:26:29.150332 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa956761-2258-481a-9906-7cb282c7ede2 service neutron] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:26:29.170988 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa956761-2258-481a-9906-7cb282c7ede2 service neutron] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:26:29.177284 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8797bf52-032c-41fa-bcc4-82dc56389dd2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:26:29.178206 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8797bf52-032c-41fa-bcc4-82dc56389dd2 service nova] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:26:29.178487 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8797bf52-032c-41fa-bcc4-82dc56389dd2 service nova] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:26:29.178760 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8797bf52-032c-41fa-bcc4-82dc56389dd2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:26:29.180895 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-8797bf52-032c-41fa-bcc4-82dc56389dd2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:26:29.194176 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8797bf52-032c-41fa-bcc4-82dc56389dd2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:26:29.195810 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8797bf52-032c-41fa-bcc4-82dc56389dd2 service nova] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:26:29.208568 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1357/2710] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:26:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:26:35.339044 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=9 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:26:35.390929 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55710084-c460-41ed-b65d-f5c13eab7eee service placement] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:26:35.403076 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55710084-c460-41ed-b65d-f5c13eab7eee service placement] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:26:35.408574 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508fdcf6-8603-4fd8-9e27-6df34028ae55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:26:35.413495 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508fdcf6-8603-4fd8-9e27-6df34028ae55 service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:26:35.413495 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508fdcf6-8603-4fd8-9e27-6df34028ae55 service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:26:35.413495 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508fdcf6-8603-4fd8-9e27-6df34028ae55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:26:35.413495 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-508fdcf6-8603-4fd8-9e27-6df34028ae55 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:26:35.424317 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508fdcf6-8603-4fd8-9e27-6df34028ae55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:26:35.426538 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508fdcf6-8603-4fd8-9e27-6df34028ae55 service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:26:35.437546 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1354/2711] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:26:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:26:35.895092 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8797bf52-032c-41fa-bcc4-82dc56389dd2 service nova] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:26:35.896732 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-8797bf52-032c-41fa-bcc4-82dc56389dd2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:26:35.963180 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8797bf52-032c-41fa-bcc4-82dc56389dd2 service nova] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:26:35.967102 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a678bb-b44d-4c98-81ad-679ee1e6f496 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:26:35.967797 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a678bb-b44d-4c98-81ad-679ee1e6f496 service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:26:35.968083 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a678bb-b44d-4c98-81ad-679ee1e6f496 service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:26:35.968361 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a678bb-b44d-4c98-81ad-679ee1e6f496 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:26:35.970117 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-27a678bb-b44d-4c98-81ad-679ee1e6f496 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:26:36.051885 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a678bb-b44d-4c98-81ad-679ee1e6f496 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:26:36.053332 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a678bb-b44d-4c98-81ad-679ee1e6f496 service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:26:36.072169 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1358/2712] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:26:35 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:26:45.433127 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=6 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:26:46.069529 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=77 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:27:19.569298 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508fdcf6-8603-4fd8-9e27-6df34028ae55 service nova] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:27:19.588785 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508fdcf6-8603-4fd8-9e27-6df34028ae55 service nova] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:27:19.592390 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea88be2-25ef-4948-a4b4-cf60e554ddad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:27:19.593024 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea88be2-25ef-4948-a4b4-cf60e554ddad service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:19.593307 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea88be2-25ef-4948-a4b4-cf60e554ddad service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:19.593632 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea88be2-25ef-4948-a4b4-cf60e554ddad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:19.595383 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-7ea88be2-25ef-4948-a4b4-cf60e554ddad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:27:19.620769 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea88be2-25ef-4948-a4b4-cf60e554ddad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:27:19.622204 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea88be2-25ef-4948-a4b4-cf60e554ddad service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:27:19.631166 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1355/2713] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:27:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:27:29.626785 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=15 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:27:30.719793 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f52a1e3-2a2e-4c8d-a2ca-dc6ec946413f None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:30.720418 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f52a1e3-2a2e-4c8d-a2ca-dc6ec946413f None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:30.720418 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f52a1e3-2a2e-4c8d-a2ca-dc6ec946413f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:30.738400 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-2f52a1e3-2a2e-4c8d-a2ca-dc6ec946413f None None] MFA Rules not processed for user `a708e1f37fd5453198e32ec6427afcaa`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:27:30.739631 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-2f52a1e3-2a2e-4c8d-a2ca-dc6ec946413f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:27:30.746564 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1359/2714] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:27:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:27:31.479912 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c517ea83-bd00-49b1-9fca-4f547fcf484d None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:31.480377 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c517ea83-bd00-49b1-9fca-4f547fcf484d None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:31.480499 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c517ea83-bd00-49b1-9fca-4f547fcf484d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:31.499837 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-c517ea83-bd00-49b1-9fca-4f547fcf484d None None] MFA Rules not processed for user `e1fc02859caf4ac09a85a4e5f962c14b`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:27:31.500963 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-c517ea83-bd00-49b1-9fca-4f547fcf484d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:27:31.588877 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1356/2715] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:27:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:27:31.593429 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f52a1e3-2a2e-4c8d-a2ca-dc6ec946413f None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:27:31.603913 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f52a1e3-2a2e-4c8d-a2ca-dc6ec946413f None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:27:31.609261 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe42090-ea33-4a33-a6e0-79972ac8e16f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:27:31.609915 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe42090-ea33-4a33-a6e0-79972ac8e16f service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:31.610195 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe42090-ea33-4a33-a6e0-79972ac8e16f service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:31.610471 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe42090-ea33-4a33-a6e0-79972ac8e16f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:31.613162 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-0fe42090-ea33-4a33-a6e0-79972ac8e16f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:27:31.626127 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe42090-ea33-4a33-a6e0-79972ac8e16f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:27:31.628420 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe42090-ea33-4a33-a6e0-79972ac8e16f service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:27:31.638906 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1360/2716] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:27:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:27:36.407712 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c517ea83-bd00-49b1-9fca-4f547fcf484d None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:27:36.409249 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-c517ea83-bd00-49b1-9fca-4f547fcf484d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:27:36.425574 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c517ea83-bd00-49b1-9fca-4f547fcf484d None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:27:36.431378 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d797f46a-1971-406e-b447-54ae530a1a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:27:36.432367 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d797f46a-1971-406e-b447-54ae530a1a38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:36.432702 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d797f46a-1971-406e-b447-54ae530a1a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:36.432983 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d797f46a-1971-406e-b447-54ae530a1a38 admin admin] PATH_INFO: `/v3/users/c243a50903b244519ccd58f02cb798da` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:36.434323 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d797f46a-1971-406e-b447-54ae530a1a38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c243a50903b244519ccd58f02cb798da)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:27:36.436339 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d797f46a-1971-406e-b447-54ae530a1a38 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:27:36.666508 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-d797f46a-1971-406e-b447-54ae530a1a38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c243a50903b244519ccd58f02cb798da'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:27:36.773372 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-d797f46a-1971-406e-b447-54ae530a1a38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c243a50903b244519ccd58f02cb798da'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:27:36.781421 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-d797f46a-1971-406e-b447-54ae530a1a38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c243a50903b244519ccd58f02cb798da'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:27:36.796827 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-d797f46a-1971-406e-b447-54ae530a1a38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c243a50903b244519ccd58f02cb798da'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:27:36.797747 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-d797f46a-1971-406e-b447-54ae530a1a38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c243a50903b244519ccd58f02cb798da'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:27:36.808361 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1357/2717] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 17:27:36 2022] DELETE /identity/v3/users/c243a50903b244519ccd58f02cb798da => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:27:37.223982 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe42090-ea33-4a33-a6e0-79972ac8e16f service neutron] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:27:37.237728 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe42090-ea33-4a33-a6e0-79972ac8e16f service neutron] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:27:37.306152 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61311bf2-2e14-4eb5-bb73-83c25b618507 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:27:37.306819 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61311bf2-2e14-4eb5-bb73-83c25b618507 service nova] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:37.307105 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61311bf2-2e14-4eb5-bb73-83c25b618507 service nova] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:37.307384 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61311bf2-2e14-4eb5-bb73-83c25b618507 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:37.309197 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-61311bf2-2e14-4eb5-bb73-83c25b618507 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:27:37.322555 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61311bf2-2e14-4eb5-bb73-83c25b618507 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:27:37.324289 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61311bf2-2e14-4eb5-bb73-83c25b618507 service nova] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:27:37.336378 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1361/2718] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:27:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:27:37.482551 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d797f46a-1971-406e-b447-54ae530a1a38 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:27:37.484230 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-d797f46a-1971-406e-b447-54ae530a1a38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:27:37.552266 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d797f46a-1971-406e-b447-54ae530a1a38 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:27:37.557195 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d662987-ed28-487d-9bbc-6958547c488d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:27:37.558077 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d662987-ed28-487d-9bbc-6958547c488d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:37.558370 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d662987-ed28-487d-9bbc-6958547c488d admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:37.558651 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d662987-ed28-487d-9bbc-6958547c488d admin admin] PATH_INFO: `/v3/projects/9e65f14a4bef42df904a5f15befe2e16` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:37.560060 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d662987-ed28-487d-9bbc-6958547c488d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e65f14a4bef42df904a5f15befe2e16)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:27:37.561956 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d662987-ed28-487d-9bbc-6958547c488d admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:27:37.584308 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:27:37.686302 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-4d662987-ed28-487d-9bbc-6958547c488d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e65f14a4bef42df904a5f15befe2e16', 'request_id': 'req-4d662987-ed28-487d-9bbc-6958547c488d'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:27:37.693135 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-4d662987-ed28-487d-9bbc-6958547c488d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e65f14a4bef42df904a5f15befe2e16', 'request_id': 'req-4d662987-ed28-487d-9bbc-6958547c488d'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:27:37.699860 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-4d662987-ed28-487d-9bbc-6958547c488d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e65f14a4bef42df904a5f15befe2e16', 'request_id': 'req-4d662987-ed28-487d-9bbc-6958547c488d'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:27:37.788373 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-4d662987-ed28-487d-9bbc-6958547c488d admin admin] The token cache is being invalidate because project 9e65f14a4bef42df904a5f15befe2e16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 17:27:37.788722 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-4d662987-ed28-487d-9bbc-6958547c488d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:27:37.789292 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-4d662987-ed28-487d-9bbc-6958547c488d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:27:37.790906 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1358/2719] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 17:27:37 2022] DELETE /identity/v3/projects/9e65f14a4bef42df904a5f15befe2e16 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:27:37.796833 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c99bf-0d3b-403b-b8bd-49a5485caf9b None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:37.797134 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c99bf-0d3b-403b-b8bd-49a5485caf9b None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:37.797417 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c99bf-0d3b-403b-b8bd-49a5485caf9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:37.817056 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-289c99bf-0d3b-403b-b8bd-49a5485caf9b None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:27:37.818262 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-289c99bf-0d3b-403b-b8bd-49a5485caf9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:27:37.879678 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1362/2720] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:27:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 17:27:37.887445 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57410fbe-3d88-4bb2-90d4-9b23a5d91d4d None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:37.887765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57410fbe-3d88-4bb2-90d4-9b23a5d91d4d None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:37.888045 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57410fbe-3d88-4bb2-90d4-9b23a5d91d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:37.906166 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-57410fbe-3d88-4bb2-90d4-9b23a5d91d4d None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:27:37.907534 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-57410fbe-3d88-4bb2-90d4-9b23a5d91d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:27:37.915546 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1359/2721] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:27:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:27:37.919696 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c99bf-0d3b-403b-b8bd-49a5485caf9b None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:27:37.930002 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c99bf-0d3b-403b-b8bd-49a5485caf9b None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:27:37.934823 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002c5b95-22f2-451b-b747-ab9226957654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:27:37.935483 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002c5b95-22f2-451b-b747-ab9226957654 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:37.935799 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002c5b95-22f2-451b-b747-ab9226957654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:37.936079 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002c5b95-22f2-451b-b747-ab9226957654 admin admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:37.936806 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002c5b95-22f2-451b-b747-ab9226957654 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 17:27:37.937111 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002c5b95-22f2-451b-b747-ab9226957654 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:27:37.938700 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002c5b95-22f2-451b-b747-ab9226957654 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:27:37.951010 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1363/2722] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 17:27:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 17:27:37.955400 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57410fbe-3d88-4bb2-90d4-9b23a5d91d4d None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:27:37.964554 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57410fbe-3d88-4bb2-90d4-9b23a5d91d4d None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:27:37.969160 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c8d0bb-5b3e-4c86-8b18-b79e2cf933cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:27:37.969801 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c8d0bb-5b3e-4c86-8b18-b79e2cf933cd admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:37.970087 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c8d0bb-5b3e-4c86-8b18-b79e2cf933cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:37.970364 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c8d0bb-5b3e-4c86-8b18-b79e2cf933cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:37.971056 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c8d0bb-5b3e-4c86-8b18-b79e2cf933cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:27:37.973078 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c8d0bb-5b3e-4c86-8b18-b79e2cf933cd admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:27:38.056788 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1360/2723] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 17:27:37 2022] POST /identity/v3/projects => generated 365 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:27:38.061123 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002c5b95-22f2-451b-b747-ab9226957654 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:27:38.071611 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002c5b95-22f2-451b-b747-ab9226957654 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:27:38.128856 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:27:38.137404 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6044d98-a4f3-4d74-8338-04f9bc4ba7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:27:38.138075 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6044d98-a4f3-4d74-8338-04f9bc4ba7e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:38.139469 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6044d98-a4f3-4d74-8338-04f9bc4ba7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:38.139469 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6044d98-a4f3-4d74-8338-04f9bc4ba7e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:38.139469 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6044d98-a4f3-4d74-8338-04f9bc4ba7e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:27:38.141251 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6044d98-a4f3-4d74-8338-04f9bc4ba7e4 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:27:38.218319 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1364/2724] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 17:27:38 2022] POST /identity/v3/users => generated 283 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:27:38.222840 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c8d0bb-5b3e-4c86-8b18-b79e2cf933cd admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:27:38.233403 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c8d0bb-5b3e-4c86-8b18-b79e2cf933cd admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:27:38.238849 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdad1cd0-00dc-484b-a0da-ba0b177cece4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:27:38.239571 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad1cd0-00dc-484b-a0da-ba0b177cece4 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:38.239849 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad1cd0-00dc-484b-a0da-ba0b177cece4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:38.240124 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad1cd0-00dc-484b-a0da-ba0b177cece4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:38.240849 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdad1cd0-00dc-484b-a0da-ba0b177cece4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:27:38.242442 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdad1cd0-00dc-484b-a0da-ba0b177cece4 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:27:38.254947 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1361/2725] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 17:27:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:27:38.258817 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6044d98-a4f3-4d74-8338-04f9bc4ba7e4 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:27:38.269494 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6044d98-a4f3-4d74-8338-04f9bc4ba7e4 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:27:38.274493 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cfb756-9c84-4546-a795-02f29ab249e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:27:38.275171 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cfb756-9c84-4546-a795-02f29ab249e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:38.275456 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cfb756-9c84-4546-a795-02f29ab249e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:38.275768 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cfb756-9c84-4546-a795-02f29ab249e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:38.276505 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cfb756-9c84-4546-a795-02f29ab249e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:27:38.278159 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cfb756-9c84-4546-a795-02f29ab249e5 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:27:38.290445 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1365/2726] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 17:27:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 17:27:38.303633 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdad1cd0-00dc-484b-a0da-ba0b177cece4 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:27:38.314181 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdad1cd0-00dc-484b-a0da-ba0b177cece4 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:27:38.319320 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372a2a34-933f-41dc-93ff-cb77f717c9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:27:38.320332 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372a2a34-933f-41dc-93ff-cb77f717c9c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:38.320685 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372a2a34-933f-41dc-93ff-cb77f717c9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:38.321035 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372a2a34-933f-41dc-93ff-cb77f717c9c8 admin admin] PATH_INFO: `/v3/projects/34ff35d092d74c71b2d8ac03f58eb058/users/89502713465a4f548a75ddb110fe0453/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:38.339521 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372a2a34-933f-41dc-93ff-cb77f717c9c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34ff35d092d74c71b2d8ac03f58eb058, user_id=89502713465a4f548a75ddb110fe0453, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:27:38.342479 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372a2a34-933f-41dc-93ff-cb77f717c9c8 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:27:38.450741 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1362/2727] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 17:27:38 2022] PUT /identity/v3/projects/34ff35d092d74c71b2d8ac03f58eb058/users/89502713465a4f548a75ddb110fe0453/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:27:38.462248 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a7077d-da0e-4608-83c6-929b164c1b0c None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:38.462534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a7077d-da0e-4608-83c6-929b164c1b0c None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:38.462809 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a7077d-da0e-4608-83c6-929b164c1b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:38.490587 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-37a7077d-da0e-4608-83c6-929b164c1b0c None None] MFA Rules not processed for user `89502713465a4f548a75ddb110fe0453`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:27:38.491948 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-37a7077d-da0e-4608-83c6-929b164c1b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:27:38.557995 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1366/2728] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:27:38 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:27:38.569599 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372a2a34-933f-41dc-93ff-cb77f717c9c8 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:27:38.571419 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-372a2a34-933f-41dc-93ff-cb77f717c9c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:27:38.629080 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372a2a34-933f-41dc-93ff-cb77f717c9c8 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:27:38.632773 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c2e67a-407e-4358-8588-0e2395560a74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:27:38.633423 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2e67a-407e-4358-8588-0e2395560a74 service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:38.633962 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2e67a-407e-4358-8588-0e2395560a74 service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:38.634245 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2e67a-407e-4358-8588-0e2395560a74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:38.642836 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c2e67a-407e-4358-8588-0e2395560a74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:27:38.644857 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c2e67a-407e-4358-8588-0e2395560a74 service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:27:38.654091 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1363/2729] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:27:38 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:27:48.554795 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=42,keystone:INSERT=3 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:27:48.624941 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a7077d-da0e-4608-83c6-929b164c1b0c None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:27:48.649792 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=69,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:27:48.688295 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a7077d-da0e-4608-83c6-929b164c1b0c None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:27:48.692948 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ec096a-eb19-4cd9-b7bc-c3041163d92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:27:48.693814 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec096a-eb19-4cd9-b7bc-c3041163d92e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:48.694095 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec096a-eb19-4cd9-b7bc-c3041163d92e admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:48.694369 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec096a-eb19-4cd9-b7bc-c3041163d92e admin admin] PATH_INFO: `/v3/users/89502713465a4f548a75ddb110fe0453` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:48.695627 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ec096a-eb19-4cd9-b7bc-c3041163d92e admin admin] RBAC: Authorizing `identity:delete_user(user_id=89502713465a4f548a75ddb110fe0453)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:27:48.699257 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ec096a-eb19-4cd9-b7bc-c3041163d92e admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:27:48.819581 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-d6ec096a-eb19-4cd9-b7bc-c3041163d92e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89502713465a4f548a75ddb110fe0453'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:27:48.863233 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-d6ec096a-eb19-4cd9-b7bc-c3041163d92e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89502713465a4f548a75ddb110fe0453'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:27:48.863825 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-d6ec096a-eb19-4cd9-b7bc-c3041163d92e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89502713465a4f548a75ddb110fe0453'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:27:48.864440 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-d6ec096a-eb19-4cd9-b7bc-c3041163d92e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89502713465a4f548a75ddb110fe0453'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:27:48.871691 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-d6ec096a-eb19-4cd9-b7bc-c3041163d92e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89502713465a4f548a75ddb110fe0453'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:27:48.890400 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1367/2730] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 17:27:48 2022] DELETE /identity/v3/users/89502713465a4f548a75ddb110fe0453 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:27:48.895829 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c1fa6c-8b2a-412a-b22a-c235d405048e None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:48.896120 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c1fa6c-8b2a-412a-b22a-c235d405048e None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:48.896403 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c1fa6c-8b2a-412a-b22a-c235d405048e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:48.914000 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-c7c1fa6c-8b2a-412a-b22a-c235d405048e None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:27:48.915192 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-c7c1fa6c-8b2a-412a-b22a-c235d405048e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:27:48.981342 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1364/2731] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:27:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 17:27:48.991189 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ec096a-eb19-4cd9-b7bc-c3041163d92e admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:27:48.992695 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-d6ec096a-eb19-4cd9-b7bc-c3041163d92e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:27:49.057030 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ec096a-eb19-4cd9-b7bc-c3041163d92e admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:27:49.060669 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6360496c-d19a-488c-b536-b40b04a7fd02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:27:49.061357 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6360496c-d19a-488c-b536-b40b04a7fd02 service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:49.061646 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6360496c-d19a-488c-b536-b40b04a7fd02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:49.061923 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6360496c-d19a-488c-b536-b40b04a7fd02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:49.071806 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6360496c-d19a-488c-b536-b40b04a7fd02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:27:49.073294 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6360496c-d19a-488c-b536-b40b04a7fd02 service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:27:49.083061 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1368/2732] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:27:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 17:27:49.475839 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c1fa6c-8b2a-412a-b22a-c235d405048e None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:27:49.477898 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-c7c1fa6c-8b2a-412a-b22a-c235d405048e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:27:49.491863 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c1fa6c-8b2a-412a-b22a-c235d405048e None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:27:49.497417 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b95a0e-62b7-45d7-bde8-3bfa760ac312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:27:49.498306 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b95a0e-62b7-45d7-bde8-3bfa760ac312 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:49.498696 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b95a0e-62b7-45d7-bde8-3bfa760ac312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:49.498811 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b95a0e-62b7-45d7-bde8-3bfa760ac312 admin admin] PATH_INFO: `/v3/projects/34ff35d092d74c71b2d8ac03f58eb058` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:49.500015 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b95a0e-62b7-45d7-bde8-3bfa760ac312 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34ff35d092d74c71b2d8ac03f58eb058)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:27:49.501951 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b95a0e-62b7-45d7-bde8-3bfa760ac312 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:27:49.607316 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-02b95a0e-62b7-45d7-bde8-3bfa760ac312 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34ff35d092d74c71b2d8ac03f58eb058', 'request_id': 'req-02b95a0e-62b7-45d7-bde8-3bfa760ac312'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:27:49.612968 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-02b95a0e-62b7-45d7-bde8-3bfa760ac312 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34ff35d092d74c71b2d8ac03f58eb058', 'request_id': 'req-02b95a0e-62b7-45d7-bde8-3bfa760ac312'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:27:49.619139 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-02b95a0e-62b7-45d7-bde8-3bfa760ac312 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34ff35d092d74c71b2d8ac03f58eb058', 'request_id': 'req-02b95a0e-62b7-45d7-bde8-3bfa760ac312'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:27:49.691781 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-02b95a0e-62b7-45d7-bde8-3bfa760ac312 admin admin] The token cache is being invalidate because project 34ff35d092d74c71b2d8ac03f58eb058 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 17:27:49.692463 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-02b95a0e-62b7-45d7-bde8-3bfa760ac312 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:27:49.692748 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-02b95a0e-62b7-45d7-bde8-3bfa760ac312 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:27:49.694305 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1365/2733] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 17:27:49 2022] DELETE /identity/v3/projects/34ff35d092d74c71b2d8ac03f58eb058 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:27:49.699185 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce0ca64-e30e-4e74-8d62-e3a28d182dd0 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:49.699632 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce0ca64-e30e-4e74-8d62-e3a28d182dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:49.699771 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce0ca64-e30e-4e74-8d62-e3a28d182dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:49.718123 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-5ce0ca64-e30e-4e74-8d62-e3a28d182dd0 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:27:49.719423 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-5ce0ca64-e30e-4e74-8d62-e3a28d182dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:27:49.736366 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:27:49.779250 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1369/2734] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:27:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:27:49.787991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5454dc-83d1-4b5a-b381-720fe2fed58a None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:49.788273 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5454dc-83d1-4b5a-b381-720fe2fed58a None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:49.788553 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5454dc-83d1-4b5a-b381-720fe2fed58a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:49.808328 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-3f5454dc-83d1-4b5a-b381-720fe2fed58a None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:27:49.810212 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-3f5454dc-83d1-4b5a-b381-720fe2fed58a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:27:49.818704 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1366/2735] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:27:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:27:49.822927 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce0ca64-e30e-4e74-8d62-e3a28d182dd0 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:27:49.832938 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce0ca64-e30e-4e74-8d62-e3a28d182dd0 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:27:49.841352 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51436495-c237-465d-86c6-be9bfa267e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:27:49.841352 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51436495-c237-465d-86c6-be9bfa267e25 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:49.841352 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51436495-c237-465d-86c6-be9bfa267e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:49.841352 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51436495-c237-465d-86c6-be9bfa267e25 admin admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:49.841787 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51436495-c237-465d-86c6-be9bfa267e25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 17:27:49.841787 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51436495-c237-465d-86c6-be9bfa267e25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:27:49.841787 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51436495-c237-465d-86c6-be9bfa267e25 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:27:49.853801 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1370/2736] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 17:27:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 17:27:49.858318 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5454dc-83d1-4b5a-b381-720fe2fed58a None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:27:49.868254 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5454dc-83d1-4b5a-b381-720fe2fed58a None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:27:49.872886 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88089310-116c-4f14-bac4-7a686e7f4de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:27:49.873560 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88089310-116c-4f14-bac4-7a686e7f4de2 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:49.873842 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88089310-116c-4f14-bac4-7a686e7f4de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:49.874118 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88089310-116c-4f14-bac4-7a686e7f4de2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:49.874831 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88089310-116c-4f14-bac4-7a686e7f4de2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:27:49.876751 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88089310-116c-4f14-bac4-7a686e7f4de2 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:27:49.967861 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1367/2737] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 17:27:49 2022] POST /identity/v3/projects => generated 387 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:27:49.972348 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51436495-c237-465d-86c6-be9bfa267e25 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:27:49.982615 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51436495-c237-465d-86c6-be9bfa267e25 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:27:50.040996 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ccfe6c-d951-45cf-a8d5-4266669858c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:27:50.041678 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ccfe6c-d951-45cf-a8d5-4266669858c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:50.041967 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ccfe6c-d951-45cf-a8d5-4266669858c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:50.042240 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ccfe6c-d951-45cf-a8d5-4266669858c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:50.042979 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ccfe6c-d951-45cf-a8d5-4266669858c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:27:50.044940 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ccfe6c-d951-45cf-a8d5-4266669858c7 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:27:50.155768 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1371/2738] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 17:27:49 2022] POST /identity/v3/users => generated 294 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:27:50.160152 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88089310-116c-4f14-bac4-7a686e7f4de2 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:27:50.171023 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88089310-116c-4f14-bac4-7a686e7f4de2 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:27:50.176068 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b4615-ba31-470d-8a0f-6b325fbab840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:27:50.176902 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b4615-ba31-470d-8a0f-6b325fbab840 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:50.181291 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b4615-ba31-470d-8a0f-6b325fbab840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:50.181291 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b4615-ba31-470d-8a0f-6b325fbab840 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:50.181291 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0b4615-ba31-470d-8a0f-6b325fbab840 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:27:50.182270 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0b4615-ba31-470d-8a0f-6b325fbab840 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:27:50.197836 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1368/2739] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 17:27:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:27:50.199210 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ccfe6c-d951-45cf-a8d5-4266669858c7 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:27:50.210474 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ccfe6c-d951-45cf-a8d5-4266669858c7 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:27:50.215684 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf0e270-1a4a-47ed-8d3e-95b0155b7b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:27:50.220644 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf0e270-1a4a-47ed-8d3e-95b0155b7b16 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:50.220644 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf0e270-1a4a-47ed-8d3e-95b0155b7b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:50.220644 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf0e270-1a4a-47ed-8d3e-95b0155b7b16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:50.220644 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf0e270-1a4a-47ed-8d3e-95b0155b7b16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:27:50.220644 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf0e270-1a4a-47ed-8d3e-95b0155b7b16 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:27:50.230138 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1372/2740] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 17:27:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:27:50.234056 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b4615-ba31-470d-8a0f-6b325fbab840 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:27:50.244612 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b4615-ba31-470d-8a0f-6b325fbab840 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:27:50.250545 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed95598-83a2-4d90-90fd-e016db7b69e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:27:50.255154 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed95598-83a2-4d90-90fd-e016db7b69e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:50.255154 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed95598-83a2-4d90-90fd-e016db7b69e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:50.255154 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed95598-83a2-4d90-90fd-e016db7b69e6 admin admin] PATH_INFO: `/v3/projects/e24f7178512a44bcb49e1403c60d5dd2/users/6d3930f1b5cb41a78c54db25fe947312/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:50.268856 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed95598-83a2-4d90-90fd-e016db7b69e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e24f7178512a44bcb49e1403c60d5dd2, user_id=6d3930f1b5cb41a78c54db25fe947312, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:27:50.271623 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed95598-83a2-4d90-90fd-e016db7b69e6 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:27:50.355618 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1369/2741] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 17:27:50 2022] PUT /identity/v3/projects/e24f7178512a44bcb49e1403c60d5dd2/users/6d3930f1b5cb41a78c54db25fe947312/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:27:50.417749 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85338dca-c28b-4b14-925d-07b037f074b8 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:50.418186 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85338dca-c28b-4b14-925d-07b037f074b8 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:50.418608 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85338dca-c28b-4b14-925d-07b037f074b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:50.448428 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-85338dca-c28b-4b14-925d-07b037f074b8 None None] MFA Rules not processed for user `6d3930f1b5cb41a78c54db25fe947312`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:27:50.451420 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-85338dca-c28b-4b14-925d-07b037f074b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:27:50.516727 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1373/2742] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:27:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:27:50.521439 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed95598-83a2-4d90-90fd-e016db7b69e6 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:27:50.542169 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:27:50.583763 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed95598-83a2-4d90-90fd-e016db7b69e6 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:27:50.588793 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff0d67-ebcb-44ae-a648-92fddf8cdb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:27:50.589538 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff0d67-ebcb-44ae-a648-92fddf8cdb03 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:50.589915 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff0d67-ebcb-44ae-a648-92fddf8cdb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:50.590271 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff0d67-ebcb-44ae-a648-92fddf8cdb03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:50.590977 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fff0d67-ebcb-44ae-a648-92fddf8cdb03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:27:50.593968 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fff0d67-ebcb-44ae-a648-92fddf8cdb03 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:27:50.693276 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1370/2743] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 17:27:50 2022] POST /identity/v3/projects => generated 385 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:27:50.697477 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85338dca-c28b-4b14-925d-07b037f074b8 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:27:50.710560 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85338dca-c28b-4b14-925d-07b037f074b8 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:27:50.724490 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:27:50.773413 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eff649-1e62-48d8-8aea-f5d954594dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:27:50.774030 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eff649-1e62-48d8-8aea-f5d954594dac admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:50.774323 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eff649-1e62-48d8-8aea-f5d954594dac admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:50.774606 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eff649-1e62-48d8-8aea-f5d954594dac admin admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:50.775326 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eff649-1e62-48d8-8aea-f5d954594dac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:27:50.777320 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eff649-1e62-48d8-8aea-f5d954594dac admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:27:50.875090 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1374/2744] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 17:27:50 2022] POST /identity/v3/users => generated 299 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 17:27:50.879755 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff0d67-ebcb-44ae-a648-92fddf8cdb03 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:27:50.890771 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff0d67-ebcb-44ae-a648-92fddf8cdb03 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:27:50.896726 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ba338-ea1d-4b5b-ba2f-448d63e7ed1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:27:50.897495 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ba338-ea1d-4b5b-ba2f-448d63e7ed1a admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:50.897865 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ba338-ea1d-4b5b-ba2f-448d63e7ed1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:50.898192 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ba338-ea1d-4b5b-ba2f-448d63e7ed1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:50.899156 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08ba338-ea1d-4b5b-ba2f-448d63e7ed1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:27:50.900895 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08ba338-ea1d-4b5b-ba2f-448d63e7ed1a admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:27:50.912495 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1371/2745] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 17:27:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:27:50.916897 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eff649-1e62-48d8-8aea-f5d954594dac admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:27:50.927766 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eff649-1e62-48d8-8aea-f5d954594dac admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:27:50.932804 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df3cb1-c1fd-471b-96f5-7bbdaf1432df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:27:50.933860 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df3cb1-c1fd-471b-96f5-7bbdaf1432df admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:50.934145 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df3cb1-c1fd-471b-96f5-7bbdaf1432df admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:50.934414 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df3cb1-c1fd-471b-96f5-7bbdaf1432df admin admin] PATH_INFO: `/v3/projects/b176b11bffe74cb3b2d1ad0ccd816e4e/users/e763388c9f034928bda923a465aa2088/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:50.950619 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62df3cb1-c1fd-471b-96f5-7bbdaf1432df admin admin] RBAC: Authorizing `identity:create_grant(project_id=b176b11bffe74cb3b2d1ad0ccd816e4e, user_id=e763388c9f034928bda923a465aa2088, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:27:50.953319 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62df3cb1-c1fd-471b-96f5-7bbdaf1432df admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:27:51.056709 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1375/2746] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 17:27:50 2022] PUT /identity/v3/projects/b176b11bffe74cb3b2d1ad0ccd816e4e/users/e763388c9f034928bda923a465aa2088/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:27:51.056929 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ba338-ea1d-4b5b-ba2f-448d63e7ed1a admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:27:51.063133 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ba338-ea1d-4b5b-ba2f-448d63e7ed1a admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:27:51.122084 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495a3647-55eb-4045-aadc-11e1897cbe9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:27:51.123005 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a3647-55eb-4045-aadc-11e1897cbe9d admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:51.123005 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a3647-55eb-4045-aadc-11e1897cbe9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:51.123346 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a3647-55eb-4045-aadc-11e1897cbe9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:51.124246 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495a3647-55eb-4045-aadc-11e1897cbe9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:27:51.125688 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495a3647-55eb-4045-aadc-11e1897cbe9d admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:27:51.136219 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1372/2747] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 17:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 17:27:51.140482 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df3cb1-c1fd-471b-96f5-7bbdaf1432df admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:27:51.151729 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df3cb1-c1fd-471b-96f5-7bbdaf1432df admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:27:51.156465 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23835be-7376-42f2-8a00-274bbebdb271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:27:51.157405 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23835be-7376-42f2-8a00-274bbebdb271 admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:51.157796 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23835be-7376-42f2-8a00-274bbebdb271 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:51.158083 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23835be-7376-42f2-8a00-274bbebdb271 admin admin] PATH_INFO: `/v3/projects/b176b11bffe74cb3b2d1ad0ccd816e4e/users/e763388c9f034928bda923a465aa2088/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:51.160218 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23835be-7376-42f2-8a00-274bbebdb271 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b176b11bffe74cb3b2d1ad0ccd816e4e, user_id=e763388c9f034928bda923a465aa2088, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:27:51.162849 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23835be-7376-42f2-8a00-274bbebdb271 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:27:51.173608 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1376/2748] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 17:27:51 2022] PUT /identity/v3/projects/b176b11bffe74cb3b2d1ad0ccd816e4e/users/e763388c9f034928bda923a465aa2088/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:27:51.184773 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3cfba9-4d0a-4553-939e-8824dc4be2fc None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:51.185063 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3cfba9-4d0a-4553-939e-8824dc4be2fc None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:51.185342 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3cfba9-4d0a-4553-939e-8824dc4be2fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:51.215913 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-6e3cfba9-4d0a-4553-939e-8824dc4be2fc None None] MFA Rules not processed for user `e763388c9f034928bda923a465aa2088`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:27:51.217135 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-6e3cfba9-4d0a-4553-939e-8824dc4be2fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:27:51.291386 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1373/2749] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:27:51 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:27:51.306536 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23835be-7376-42f2-8a00-274bbebdb271 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:27:51.308364 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-c23835be-7376-42f2-8a00-274bbebdb271 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:27:51.380218 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23835be-7376-42f2-8a00-274bbebdb271 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:27:51.384358 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae8a846-9c4c-4e5a-b0fa-39fbac8e8705 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:27:51.385028 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8a846-9c4c-4e5a-b0fa-39fbac8e8705 service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:51.385311 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8a846-9c4c-4e5a-b0fa-39fbac8e8705 service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:51.385595 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8a846-9c4c-4e5a-b0fa-39fbac8e8705 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:51.395374 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae8a846-9c4c-4e5a-b0fa-39fbac8e8705 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:27:51.396879 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae8a846-9c4c-4e5a-b0fa-39fbac8e8705 service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:27:51.407520 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1377/2750] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:27:51 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:27:51.475998 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3cfba9-4d0a-4553-939e-8824dc4be2fc None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:27:51.497621 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:27:51.557148 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3cfba9-4d0a-4553-939e-8824dc4be2fc None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:27:51.562850 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d53d9-1fc0-4beb-ad07-ae815ae8c9c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:27:51.563681 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d53d9-1fc0-4beb-ad07-ae815ae8c9c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:51.563956 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d53d9-1fc0-4beb-ad07-ae815ae8c9c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:51.564225 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d53d9-1fc0-4beb-ad07-ae815ae8c9c9 admin admin] PATH_INFO: `/v3/users/6d3930f1b5cb41a78c54db25fe947312` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:51.565637 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66d53d9-1fc0-4beb-ad07-ae815ae8c9c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d3930f1b5cb41a78c54db25fe947312)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:27:51.567787 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66d53d9-1fc0-4beb-ad07-ae815ae8c9c9 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:27:51.760289 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-e66d53d9-1fc0-4beb-ad07-ae815ae8c9c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d3930f1b5cb41a78c54db25fe947312'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:27:51.852712 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-e66d53d9-1fc0-4beb-ad07-ae815ae8c9c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d3930f1b5cb41a78c54db25fe947312'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:27:51.860762 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-e66d53d9-1fc0-4beb-ad07-ae815ae8c9c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d3930f1b5cb41a78c54db25fe947312'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:27:51.877960 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-e66d53d9-1fc0-4beb-ad07-ae815ae8c9c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d3930f1b5cb41a78c54db25fe947312'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:27:51.878812 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-e66d53d9-1fc0-4beb-ad07-ae815ae8c9c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d3930f1b5cb41a78c54db25fe947312'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:27:51.885684 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1374/2751] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 17:27:51 2022] DELETE /identity/v3/users/6d3930f1b5cb41a78c54db25fe947312 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:27:51.890956 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662f96e-e9af-4ecf-9304-9bec5bcb3e2b None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:51.891537 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662f96e-e9af-4ecf-9304-9bec5bcb3e2b None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:51.891647 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662f96e-e9af-4ecf-9304-9bec5bcb3e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:51.916014 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-9662f96e-e9af-4ecf-9304-9bec5bcb3e2b None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:27:51.916925 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-9662f96e-e9af-4ecf-9304-9bec5bcb3e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:27:51.989295 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1378/2752] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:27:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:27:51.999898 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d53d9-1fc0-4beb-ad07-ae815ae8c9c9 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:27:52.001325 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e66d53d9-1fc0-4beb-ad07-ae815ae8c9c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:27:52.065394 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d53d9-1fc0-4beb-ad07-ae815ae8c9c9 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:27:52.070617 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe57adb-07d3-4c3d-8eee-67f839a67aa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:27:52.071305 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe57adb-07d3-4c3d-8eee-67f839a67aa0 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:52.071630 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe57adb-07d3-4c3d-8eee-67f839a67aa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:52.072515 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe57adb-07d3-4c3d-8eee-67f839a67aa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:52.084715 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe57adb-07d3-4c3d-8eee-67f839a67aa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:27:52.086076 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe57adb-07d3-4c3d-8eee-67f839a67aa0 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:27:52.099356 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1375/2753] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:27:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:27:52.820602 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662f96e-e9af-4ecf-9304-9bec5bcb3e2b None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:27:52.825099 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-9662f96e-e9af-4ecf-9304-9bec5bcb3e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:27:52.836746 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662f96e-e9af-4ecf-9304-9bec5bcb3e2b None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:27:52.843388 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8722ba87-ccdd-4742-9889-597b8cc8f6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:27:52.843977 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8722ba87-ccdd-4742-9889-597b8cc8f6e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:52.844359 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8722ba87-ccdd-4742-9889-597b8cc8f6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:52.844735 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8722ba87-ccdd-4742-9889-597b8cc8f6e7 admin admin] PATH_INFO: `/v3/projects/e24f7178512a44bcb49e1403c60d5dd2` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:52.847116 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8722ba87-ccdd-4742-9889-597b8cc8f6e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e24f7178512a44bcb49e1403c60d5dd2)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:27:52.851729 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8722ba87-ccdd-4742-9889-597b8cc8f6e7 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:27:52.863710 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:27:52.959015 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-8722ba87-ccdd-4742-9889-597b8cc8f6e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e24f7178512a44bcb49e1403c60d5dd2', 'request_id': 'req-8722ba87-ccdd-4742-9889-597b8cc8f6e7'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:27:52.967430 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-8722ba87-ccdd-4742-9889-597b8cc8f6e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e24f7178512a44bcb49e1403c60d5dd2', 'request_id': 'req-8722ba87-ccdd-4742-9889-597b8cc8f6e7'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:27:53.012747 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-8722ba87-ccdd-4742-9889-597b8cc8f6e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e24f7178512a44bcb49e1403c60d5dd2', 'request_id': 'req-8722ba87-ccdd-4742-9889-597b8cc8f6e7'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:27:53.029574 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-8722ba87-ccdd-4742-9889-597b8cc8f6e7 admin admin] The token cache is being invalidate because project e24f7178512a44bcb49e1403c60d5dd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 17:27:53.029889 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-8722ba87-ccdd-4742-9889-597b8cc8f6e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:27:53.030567 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-8722ba87-ccdd-4742-9889-597b8cc8f6e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:27:53.032761 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1379/2754] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 17:27:52 2022] DELETE /identity/v3/projects/e24f7178512a44bcb49e1403c60d5dd2 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:27:53.037325 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe57adb-07d3-4c3d-8eee-67f839a67aa0 service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:27:53.048049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe57adb-07d3-4c3d-8eee-67f839a67aa0 service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:27:53.085141 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:27:53.113353 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103c159b-0385-4011-8782-9693212432de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:27:53.116032 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103c159b-0385-4011-8782-9693212432de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:53.116032 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103c159b-0385-4011-8782-9693212432de admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:53.116032 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103c159b-0385-4011-8782-9693212432de admin admin] PATH_INFO: `/v3/users/e763388c9f034928bda923a465aa2088` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:53.116386 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103c159b-0385-4011-8782-9693212432de admin admin] RBAC: Authorizing `identity:delete_user(user_id=e763388c9f034928bda923a465aa2088)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:27:53.118328 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103c159b-0385-4011-8782-9693212432de admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:27:53.316290 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-103c159b-0385-4011-8782-9693212432de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e763388c9f034928bda923a465aa2088'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:27:53.383938 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-103c159b-0385-4011-8782-9693212432de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e763388c9f034928bda923a465aa2088'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:27:53.391402 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-103c159b-0385-4011-8782-9693212432de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e763388c9f034928bda923a465aa2088'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:27:53.403690 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-103c159b-0385-4011-8782-9693212432de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e763388c9f034928bda923a465aa2088'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:27:53.404409 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-103c159b-0385-4011-8782-9693212432de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e763388c9f034928bda923a465aa2088'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:27:53.410519 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1376/2755] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 17:27:53 2022] DELETE /identity/v3/users/e763388c9f034928bda923a465aa2088 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:27:53.865653 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8722ba87-ccdd-4742-9889-597b8cc8f6e7 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:27:53.872806 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-8722ba87-ccdd-4742-9889-597b8cc8f6e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:27:53.886359 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8722ba87-ccdd-4742-9889-597b8cc8f6e7 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:27:53.946291 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a29177b-6110-4ed7-9674-8301ea41f27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:27:53.947165 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a29177b-6110-4ed7-9674-8301ea41f27f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:53.947821 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a29177b-6110-4ed7-9674-8301ea41f27f admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:53.948105 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a29177b-6110-4ed7-9674-8301ea41f27f admin admin] PATH_INFO: `/v3/projects/b176b11bffe74cb3b2d1ad0ccd816e4e` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:53.949316 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a29177b-6110-4ed7-9674-8301ea41f27f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b176b11bffe74cb3b2d1ad0ccd816e4e)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:27:53.951239 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a29177b-6110-4ed7-9674-8301ea41f27f admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:27:54.058894 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-7a29177b-6110-4ed7-9674-8301ea41f27f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b176b11bffe74cb3b2d1ad0ccd816e4e', 'request_id': 'req-7a29177b-6110-4ed7-9674-8301ea41f27f'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:27:54.065424 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-7a29177b-6110-4ed7-9674-8301ea41f27f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b176b11bffe74cb3b2d1ad0ccd816e4e', 'request_id': 'req-7a29177b-6110-4ed7-9674-8301ea41f27f'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:27:54.126340 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-7a29177b-6110-4ed7-9674-8301ea41f27f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b176b11bffe74cb3b2d1ad0ccd816e4e', 'request_id': 'req-7a29177b-6110-4ed7-9674-8301ea41f27f'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:27:54.137811 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-7a29177b-6110-4ed7-9674-8301ea41f27f admin admin] The token cache is being invalidate because project b176b11bffe74cb3b2d1ad0ccd816e4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 17:27:54.141607 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-7a29177b-6110-4ed7-9674-8301ea41f27f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:27:54.141607 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-7a29177b-6110-4ed7-9674-8301ea41f27f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:27:54.141607 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1380/2756] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 17:27:53 2022] DELETE /identity/v3/projects/b176b11bffe74cb3b2d1ad0ccd816e4e => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:27:54.148258 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240d1025-4c90-4470-8cc2-aa23eaca7791 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:54.148806 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240d1025-4c90-4470-8cc2-aa23eaca7791 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:54.151023 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240d1025-4c90-4470-8cc2-aa23eaca7791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:54.171942 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-240d1025-4c90-4470-8cc2-aa23eaca7791 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:27:54.173123 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-240d1025-4c90-4470-8cc2-aa23eaca7791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:27:54.234926 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1377/2757] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:27:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:27:54.243108 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbbc30c-8065-47af-9a4e-bc07f86ceb87 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:54.243399 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbbc30c-8065-47af-9a4e-bc07f86ceb87 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:54.243812 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbbc30c-8065-47af-9a4e-bc07f86ceb87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:54.261985 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-8fbbc30c-8065-47af-9a4e-bc07f86ceb87 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:27:54.263194 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-8fbbc30c-8065-47af-9a4e-bc07f86ceb87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:27:54.271657 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1381/2758] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:27:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:27:54.276028 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240d1025-4c90-4470-8cc2-aa23eaca7791 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:27:54.286208 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240d1025-4c90-4470-8cc2-aa23eaca7791 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:27:54.290680 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b2d9ea-59c0-4785-96f2-80dacb88c531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:27:54.291327 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b2d9ea-59c0-4785-96f2-80dacb88c531 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:54.291641 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b2d9ea-59c0-4785-96f2-80dacb88c531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:54.291922 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b2d9ea-59c0-4785-96f2-80dacb88c531 admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:54.292709 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b2d9ea-59c0-4785-96f2-80dacb88c531 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 17:27:54.293188 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b2d9ea-59c0-4785-96f2-80dacb88c531 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:27:54.294527 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b2d9ea-59c0-4785-96f2-80dacb88c531 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:27:54.306510 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1378/2759] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 17:27:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 17:27:54.310767 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbbc30c-8065-47af-9a4e-bc07f86ceb87 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:27:54.320050 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbbc30c-8065-47af-9a4e-bc07f86ceb87 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:27:54.324637 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d97980-36e6-4f3b-bee4-010b3632a3e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:27:54.325278 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d97980-36e6-4f3b-bee4-010b3632a3e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:54.325564 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d97980-36e6-4f3b-bee4-010b3632a3e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:54.325834 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d97980-36e6-4f3b-bee4-010b3632a3e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:54.326525 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d97980-36e6-4f3b-bee4-010b3632a3e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:27:54.328448 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d97980-36e6-4f3b-bee4-010b3632a3e2 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:27:54.383661 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:27:54.390731 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1382/2760] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 17:27:54 2022] POST /identity/v3/projects => generated 379 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:27:54.395394 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b2d9ea-59c0-4785-96f2-80dacb88c531 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:27:54.405370 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b2d9ea-59c0-4785-96f2-80dacb88c531 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:27:54.447809 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:27:54.463072 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d1a5ad-3034-42fa-aa12-c88d6b8766af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:27:54.463776 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d1a5ad-3034-42fa-aa12-c88d6b8766af admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:54.464063 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d1a5ad-3034-42fa-aa12-c88d6b8766af admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:54.464339 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d1a5ad-3034-42fa-aa12-c88d6b8766af admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:54.465034 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d1a5ad-3034-42fa-aa12-c88d6b8766af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:27:54.466970 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d1a5ad-3034-42fa-aa12-c88d6b8766af admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:27:54.526330 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1379/2761] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 17:27:54 2022] POST /identity/v3/users => generated 290 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:27:54.530530 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d97980-36e6-4f3b-bee4-010b3632a3e2 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:27:54.541649 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d97980-36e6-4f3b-bee4-010b3632a3e2 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:27:54.546312 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d586b74-2ce7-4133-9ba7-5acfba18965c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:27:54.546971 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d586b74-2ce7-4133-9ba7-5acfba18965c admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:54.547257 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d586b74-2ce7-4133-9ba7-5acfba18965c admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:54.547570 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d586b74-2ce7-4133-9ba7-5acfba18965c admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:54.549867 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d586b74-2ce7-4133-9ba7-5acfba18965c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:27:54.550124 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d586b74-2ce7-4133-9ba7-5acfba18965c admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:27:54.560948 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1383/2762] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 17:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:27:54.565356 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d1a5ad-3034-42fa-aa12-c88d6b8766af admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:27:54.575261 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d1a5ad-3034-42fa-aa12-c88d6b8766af admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:27:54.579926 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc53dd79-e8d6-4ff5-bc1c-edca1059273b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:27:54.581229 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc53dd79-e8d6-4ff5-bc1c-edca1059273b admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:54.581513 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc53dd79-e8d6-4ff5-bc1c-edca1059273b admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:54.581793 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc53dd79-e8d6-4ff5-bc1c-edca1059273b admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:54.582520 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc53dd79-e8d6-4ff5-bc1c-edca1059273b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:27:54.584153 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc53dd79-e8d6-4ff5-bc1c-edca1059273b admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:27:54.596913 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1380/2763] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 17:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:27:54.599883 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d586b74-2ce7-4133-9ba7-5acfba18965c admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:27:54.610190 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d586b74-2ce7-4133-9ba7-5acfba18965c admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:27:54.614865 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafad6a9-4d17-4aed-8d5c-5006bfc0eff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:27:54.615805 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafad6a9-4d17-4aed-8d5c-5006bfc0eff6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:54.616086 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafad6a9-4d17-4aed-8d5c-5006bfc0eff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:54.616372 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafad6a9-4d17-4aed-8d5c-5006bfc0eff6 admin admin] PATH_INFO: `/v3/projects/6b2627c92e3c4e53b91dfa0f8860ac6c/users/f5c673778792496ab6e5855a445bd15d/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:54.632573 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafad6a9-4d17-4aed-8d5c-5006bfc0eff6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b2627c92e3c4e53b91dfa0f8860ac6c, user_id=f5c673778792496ab6e5855a445bd15d, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:27:54.635202 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafad6a9-4d17-4aed-8d5c-5006bfc0eff6 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:27:54.703784 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1384/2764] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 17:27:54 2022] PUT /identity/v3/projects/6b2627c92e3c4e53b91dfa0f8860ac6c/users/f5c673778792496ab6e5855a445bd15d/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:27:54.715766 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b24b1-b6ca-4263-a0f4-e2cff8633d4e None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:54.716082 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b24b1-b6ca-4263-a0f4-e2cff8633d4e None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:54.716359 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b24b1-b6ca-4263-a0f4-e2cff8633d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:54.742624 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-264b24b1-b6ca-4263-a0f4-e2cff8633d4e None None] MFA Rules not processed for user `f5c673778792496ab6e5855a445bd15d`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:27:54.743855 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-264b24b1-b6ca-4263-a0f4-e2cff8633d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:27:54.808329 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1381/2765] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:27:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:27:54.812988 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafad6a9-4d17-4aed-8d5c-5006bfc0eff6 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:27:54.876114 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafad6a9-4d17-4aed-8d5c-5006bfc0eff6 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:27:54.881649 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ea9005-bf6d-4359-a70c-6a8f16d157a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:27:54.882316 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea9005-bf6d-4359-a70c-6a8f16d157a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:54.882619 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea9005-bf6d-4359-a70c-6a8f16d157a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:54.882898 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea9005-bf6d-4359-a70c-6a8f16d157a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:54.883624 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ea9005-bf6d-4359-a70c-6a8f16d157a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:27:54.885549 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ea9005-bf6d-4359-a70c-6a8f16d157a2 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:27:54.990241 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1385/2766] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 17:27:54 2022] POST /identity/v3/projects => generated 377 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:27:54.994342 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264b24b1-b6ca-4263-a0f4-e2cff8633d4e None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:27:55.005812 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264b24b1-b6ca-4263-a0f4-e2cff8633d4e None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:27:55.061445 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a2a1f4-8010-423a-ac08-b61931432082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:27:55.062038 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2a1f4-8010-423a-ac08-b61931432082 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:55.062323 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2a1f4-8010-423a-ac08-b61931432082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:55.062604 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2a1f4-8010-423a-ac08-b61931432082 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:55.063295 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a2a1f4-8010-423a-ac08-b61931432082 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:27:55.065883 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a2a1f4-8010-423a-ac08-b61931432082 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:27:55.137665 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1382/2767] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 17:27:54 2022] POST /identity/v3/users => generated 289 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:27:55.141965 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ea9005-bf6d-4359-a70c-6a8f16d157a2 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:27:55.152835 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ea9005-bf6d-4359-a70c-6a8f16d157a2 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:27:55.157440 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a6d5f4-c5a2-4204-ad3e-0b4f074d2eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:27:55.158105 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6d5f4-c5a2-4204-ad3e-0b4f074d2eae admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:55.158556 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6d5f4-c5a2-4204-ad3e-0b4f074d2eae admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:55.158942 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6d5f4-c5a2-4204-ad3e-0b4f074d2eae admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:55.159698 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a6d5f4-c5a2-4204-ad3e-0b4f074d2eae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:27:55.161299 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a6d5f4-c5a2-4204-ad3e-0b4f074d2eae admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:27:55.171870 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1386/2768] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 17:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:27:55.176404 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a2a1f4-8010-423a-ac08-b61931432082 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:27:55.186962 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a2a1f4-8010-423a-ac08-b61931432082 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:27:55.192482 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e14387-63f2-4253-9965-47041a4c091e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:27:55.193135 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e14387-63f2-4253-9965-47041a4c091e admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:55.193417 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e14387-63f2-4253-9965-47041a4c091e admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:55.193738 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e14387-63f2-4253-9965-47041a4c091e admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:55.194463 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e14387-63f2-4253-9965-47041a4c091e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:27:55.196096 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e14387-63f2-4253-9965-47041a4c091e admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:27:55.206339 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1383/2769] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 17:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:27:55.210375 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a6d5f4-c5a2-4204-ad3e-0b4f074d2eae admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:27:55.220347 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a6d5f4-c5a2-4204-ad3e-0b4f074d2eae admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:27:55.226720 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e60de51-9b7d-4d96-af5a-bfb6727bb279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:27:55.226720 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e60de51-9b7d-4d96-af5a-bfb6727bb279 admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:55.226720 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e60de51-9b7d-4d96-af5a-bfb6727bb279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:55.226720 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e60de51-9b7d-4d96-af5a-bfb6727bb279 admin admin] PATH_INFO: `/v3/projects/e3d1195d7adb440eabb981c535dbb282/users/a6eb1f3996b34589b57f3ac0730e08ab/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:55.241720 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e60de51-9b7d-4d96-af5a-bfb6727bb279 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3d1195d7adb440eabb981c535dbb282, user_id=a6eb1f3996b34589b57f3ac0730e08ab, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:27:55.244429 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e60de51-9b7d-4d96-af5a-bfb6727bb279 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:27:55.309959 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1387/2770] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 17:27:55 2022] PUT /identity/v3/projects/e3d1195d7adb440eabb981c535dbb282/users/a6eb1f3996b34589b57f3ac0730e08ab/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 17:27:55.321119 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecad8a2-ff90-4678-8ce8-cf8c9471ea16 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:55.321341 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecad8a2-ff90-4678-8ce8-cf8c9471ea16 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:55.321621 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecad8a2-ff90-4678-8ce8-cf8c9471ea16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:55.348291 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-cecad8a2-ff90-4678-8ce8-cf8c9471ea16 None None] MFA Rules not processed for user `a6eb1f3996b34589b57f3ac0730e08ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:27:55.349487 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-cecad8a2-ff90-4678-8ce8-cf8c9471ea16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:27:55.367332 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:27:55.413468 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1384/2771] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:27:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:27:55.418010 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e60de51-9b7d-4d96-af5a-bfb6727bb279 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:27:55.479953 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e60de51-9b7d-4d96-af5a-bfb6727bb279 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:27:55.484570 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d27c21-0d43-4e11-98c1-ef150117fa77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:27:55.485230 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d27c21-0d43-4e11-98c1-ef150117fa77 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:55.485514 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d27c21-0d43-4e11-98c1-ef150117fa77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:55.485793 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d27c21-0d43-4e11-98c1-ef150117fa77 admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:55.486614 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d27c21-0d43-4e11-98c1-ef150117fa77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:27:55.488524 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d27c21-0d43-4e11-98c1-ef150117fa77 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:27:55.567289 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:27:55.576726 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1388/2772] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 17:27:55 2022] POST /identity/v3/projects => generated 375 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:27:55.578784 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecad8a2-ff90-4678-8ce8-cf8c9471ea16 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:27:55.590320 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecad8a2-ff90-4678-8ce8-cf8c9471ea16 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:27:55.646356 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43da853c-a239-4ad4-9af3-0df994479d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:27:55.647021 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da853c-a239-4ad4-9af3-0df994479d70 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:55.647306 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da853c-a239-4ad4-9af3-0df994479d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:55.647614 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da853c-a239-4ad4-9af3-0df994479d70 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:55.648312 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43da853c-a239-4ad4-9af3-0df994479d70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:27:55.650226 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43da853c-a239-4ad4-9af3-0df994479d70 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:27:55.744471 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1385/2773] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 17:27:55 2022] POST /identity/v3/users => generated 294 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:27:55.748427 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d27c21-0d43-4e11-98c1-ef150117fa77 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:27:55.759290 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d27c21-0d43-4e11-98c1-ef150117fa77 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:27:55.763936 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603942f7-8198-49e5-9d0d-2a08699a7289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:27:55.764605 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603942f7-8198-49e5-9d0d-2a08699a7289 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:55.764924 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603942f7-8198-49e5-9d0d-2a08699a7289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:55.765209 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603942f7-8198-49e5-9d0d-2a08699a7289 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:55.765936 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603942f7-8198-49e5-9d0d-2a08699a7289 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:27:55.767565 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603942f7-8198-49e5-9d0d-2a08699a7289 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:27:55.779333 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1389/2774] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 17:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:27:55.783731 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43da853c-a239-4ad4-9af3-0df994479d70 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:27:55.793846 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43da853c-a239-4ad4-9af3-0df994479d70 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:27:55.798450 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f76f3-37a1-4563-966a-bc887ba88e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:27:55.800717 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f76f3-37a1-4563-966a-bc887ba88e10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:55.801013 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f76f3-37a1-4563-966a-bc887ba88e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:55.801291 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f76f3-37a1-4563-966a-bc887ba88e10 admin admin] PATH_INFO: `/v3/projects/01ac0e5c57444334a3215c2cdf35b16c/users/612ecf415f994674895e2a404cdbd318/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:55.818655 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18f76f3-37a1-4563-966a-bc887ba88e10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01ac0e5c57444334a3215c2cdf35b16c, user_id=612ecf415f994674895e2a404cdbd318, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:27:55.821699 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18f76f3-37a1-4563-966a-bc887ba88e10 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:27:55.875780 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1386/2775] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 17:27:55 2022] PUT /identity/v3/projects/01ac0e5c57444334a3215c2cdf35b16c/users/612ecf415f994674895e2a404cdbd318/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:27:55.880484 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603942f7-8198-49e5-9d0d-2a08699a7289 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:27:55.890640 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603942f7-8198-49e5-9d0d-2a08699a7289 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:27:55.946121 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63bc710-9cbd-4495-b2a4-c780836af184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:27:55.946782 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63bc710-9cbd-4495-b2a4-c780836af184 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:55.947065 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63bc710-9cbd-4495-b2a4-c780836af184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:55.947360 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63bc710-9cbd-4495-b2a4-c780836af184 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:55.948103 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63bc710-9cbd-4495-b2a4-c780836af184 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:27:55.949687 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63bc710-9cbd-4495-b2a4-c780836af184 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:27:55.959249 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1390/2776] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 17:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:27:55.963353 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f76f3-37a1-4563-966a-bc887ba88e10 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:27:55.974134 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f76f3-37a1-4563-966a-bc887ba88e10 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:27:55.978713 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5d7c92-d2e7-4224-aa3b-efea3c20db33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:27:55.979636 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d7c92-d2e7-4224-aa3b-efea3c20db33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:55.979916 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d7c92-d2e7-4224-aa3b-efea3c20db33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:55.980186 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d7c92-d2e7-4224-aa3b-efea3c20db33 admin admin] PATH_INFO: `/v3/projects/01ac0e5c57444334a3215c2cdf35b16c/users/612ecf415f994674895e2a404cdbd318/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:55.982155 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5d7c92-d2e7-4224-aa3b-efea3c20db33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01ac0e5c57444334a3215c2cdf35b16c, user_id=612ecf415f994674895e2a404cdbd318, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:27:55.984793 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5d7c92-d2e7-4224-aa3b-efea3c20db33 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:27:55.995410 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1387/2777] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 17:27:55 2022] PUT /identity/v3/projects/01ac0e5c57444334a3215c2cdf35b16c/users/612ecf415f994674895e2a404cdbd318/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:27:56.006164 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d47f2-79b0-4607-9e3e-49d82f17c945 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:56.006451 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d47f2-79b0-4607-9e3e-49d82f17c945 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:56.006820 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d47f2-79b0-4607-9e3e-49d82f17c945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:56.032209 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-829d47f2-79b0-4607-9e3e-49d82f17c945 None None] MFA Rules not processed for user `612ecf415f994674895e2a404cdbd318`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:27:56.033947 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-829d47f2-79b0-4607-9e3e-49d82f17c945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:27:56.102934 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1391/2778] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:27:56 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:27:56.114797 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5d7c92-d2e7-4224-aa3b-efea3c20db33 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:27:56.116790 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-0d5d7c92-d2e7-4224-aa3b-efea3c20db33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:27:56.175188 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5d7c92-d2e7-4224-aa3b-efea3c20db33 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:27:56.178865 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e98829d-a420-48b2-99f0-09d4f3afd068 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:27:56.183402 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e98829d-a420-48b2-99f0-09d4f3afd068 service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:56.183402 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e98829d-a420-48b2-99f0-09d4f3afd068 service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:56.183402 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e98829d-a420-48b2-99f0-09d4f3afd068 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:56.190136 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e98829d-a420-48b2-99f0-09d4f3afd068 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:27:56.191605 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e98829d-a420-48b2-99f0-09d4f3afd068 service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:27:56.200938 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1388/2779] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:27:56 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:27:56.305563 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829d47f2-79b0-4607-9e3e-49d82f17c945 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:27:56.307245 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-829d47f2-79b0-4607-9e3e-49d82f17c945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:27:56.320565 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829d47f2-79b0-4607-9e3e-49d82f17c945 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:27:56.324308 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef5f02f-c6cc-4c09-861f-3cb06bbc0baa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:27:56.324965 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef5f02f-c6cc-4c09-861f-3cb06bbc0baa service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:27:56.325253 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef5f02f-c6cc-4c09-861f-3cb06bbc0baa service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:27:56.325534 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef5f02f-c6cc-4c09-861f-3cb06bbc0baa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:27:56.382022 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef5f02f-c6cc-4c09-861f-3cb06bbc0baa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:27:56.383354 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef5f02f-c6cc-4c09-861f-3cb06bbc0baa service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:27:56.392207 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1392/2780] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:27:56 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:28:00.136005 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e98829d-a420-48b2-99f0-09d4f3afd068 service cinder] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:28:00.153866 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:28:00.206247 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e98829d-a420-48b2-99f0-09d4f3afd068 service cinder] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:28:00.210790 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979bc792-a3dd-4639-895c-9801c99df128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:28:00.211669 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979bc792-a3dd-4639-895c-9801c99df128 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:00.214811 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979bc792-a3dd-4639-895c-9801c99df128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:00.214811 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979bc792-a3dd-4639-895c-9801c99df128 admin admin] PATH_INFO: `/v3/users/f5c673778792496ab6e5855a445bd15d` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:00.214811 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979bc792-a3dd-4639-895c-9801c99df128 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5c673778792496ab6e5855a445bd15d)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:28:00.215315 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979bc792-a3dd-4639-895c-9801c99df128 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:28:00.380630 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-979bc792-a3dd-4639-895c-9801c99df128 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5c673778792496ab6e5855a445bd15d'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:00.435259 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-979bc792-a3dd-4639-895c-9801c99df128 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5c673778792496ab6e5855a445bd15d'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:00.443121 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-979bc792-a3dd-4639-895c-9801c99df128 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5c673778792496ab6e5855a445bd15d'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:00.455937 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-979bc792-a3dd-4639-895c-9801c99df128 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5c673778792496ab6e5855a445bd15d'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:00.456583 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-979bc792-a3dd-4639-895c-9801c99df128 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5c673778792496ab6e5855a445bd15d'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:00.462447 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1389/2781] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 17:28:00 2022] DELETE /identity/v3/users/f5c673778792496ab6e5855a445bd15d => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:28:00.466963 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21813294-9ea9-4bc1-8bf4-9525aa9eeb30 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:00.467451 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21813294-9ea9-4bc1-8bf4-9525aa9eeb30 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:00.467593 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21813294-9ea9-4bc1-8bf4-9525aa9eeb30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:00.479532 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=100 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:28:00.485233 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-21813294-9ea9-4bc1-8bf4-9525aa9eeb30 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:28:00.486970 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-21813294-9ea9-4bc1-8bf4-9525aa9eeb30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:28:00.546295 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1393/2782] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:28:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:28:00.556799 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979bc792-a3dd-4639-895c-9801c99df128 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:28:00.558188 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-979bc792-a3dd-4639-895c-9801c99df128 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:28:00.614632 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979bc792-a3dd-4639-895c-9801c99df128 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:28:00.618452 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5c2ada-9f4a-4dce-840c-bfd0a2661527 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:28:00.619099 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5c2ada-9f4a-4dce-840c-bfd0a2661527 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:00.619417 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5c2ada-9f4a-4dce-840c-bfd0a2661527 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:00.619728 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5c2ada-9f4a-4dce-840c-bfd0a2661527 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:00.629104 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5c2ada-9f4a-4dce-840c-bfd0a2661527 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:28:00.630534 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5c2ada-9f4a-4dce-840c-bfd0a2661527 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:28:00.640392 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1390/2783] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:28:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:28:01.036892 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21813294-9ea9-4bc1-8bf4-9525aa9eeb30 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:28:01.041901 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-21813294-9ea9-4bc1-8bf4-9525aa9eeb30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:28:01.050137 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21813294-9ea9-4bc1-8bf4-9525aa9eeb30 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:28:01.054958 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37afd678-8219-4de2-ac76-e217ae9c2c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:28:01.056167 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37afd678-8219-4de2-ac76-e217ae9c2c46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:01.056427 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37afd678-8219-4de2-ac76-e217ae9c2c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:01.056652 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37afd678-8219-4de2-ac76-e217ae9c2c46 admin admin] PATH_INFO: `/v3/projects/6b2627c92e3c4e53b91dfa0f8860ac6c` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:01.057870 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37afd678-8219-4de2-ac76-e217ae9c2c46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b2627c92e3c4e53b91dfa0f8860ac6c)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:28:01.059816 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37afd678-8219-4de2-ac76-e217ae9c2c46 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:28:01.183093 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-37afd678-8219-4de2-ac76-e217ae9c2c46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b2627c92e3c4e53b91dfa0f8860ac6c', 'request_id': 'req-37afd678-8219-4de2-ac76-e217ae9c2c46'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:01.189203 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-37afd678-8219-4de2-ac76-e217ae9c2c46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b2627c92e3c4e53b91dfa0f8860ac6c', 'request_id': 'req-37afd678-8219-4de2-ac76-e217ae9c2c46'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:01.251141 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-37afd678-8219-4de2-ac76-e217ae9c2c46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b2627c92e3c4e53b91dfa0f8860ac6c', 'request_id': 'req-37afd678-8219-4de2-ac76-e217ae9c2c46'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:01.262020 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-37afd678-8219-4de2-ac76-e217ae9c2c46 admin admin] The token cache is being invalidate because project 6b2627c92e3c4e53b91dfa0f8860ac6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 17:28:01.262304 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-37afd678-8219-4de2-ac76-e217ae9c2c46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:01.262920 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-37afd678-8219-4de2-ac76-e217ae9c2c46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:01.264493 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1394/2784] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 17:28:01 2022] DELETE /identity/v3/projects/6b2627c92e3c4e53b91dfa0f8860ac6c => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:28:01.268911 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5c2ada-9f4a-4dce-840c-bfd0a2661527 service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:28:01.278338 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5c2ada-9f4a-4dce-840c-bfd0a2661527 service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:28:01.310647 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:28:01.334334 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ab5867-9b59-4994-a19a-79b61ae5eb1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:28:01.335197 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab5867-9b59-4994-a19a-79b61ae5eb1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:01.335478 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab5867-9b59-4994-a19a-79b61ae5eb1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:01.335784 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab5867-9b59-4994-a19a-79b61ae5eb1d admin admin] PATH_INFO: `/v3/users/a6eb1f3996b34589b57f3ac0730e08ab` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:01.336978 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ab5867-9b59-4994-a19a-79b61ae5eb1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6eb1f3996b34589b57f3ac0730e08ab)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:28:01.338885 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ab5867-9b59-4994-a19a-79b61ae5eb1d admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:28:01.511972 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-a4ab5867-9b59-4994-a19a-79b61ae5eb1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6eb1f3996b34589b57f3ac0730e08ab'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:01.566312 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-a4ab5867-9b59-4994-a19a-79b61ae5eb1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6eb1f3996b34589b57f3ac0730e08ab'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:01.573775 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-a4ab5867-9b59-4994-a19a-79b61ae5eb1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6eb1f3996b34589b57f3ac0730e08ab'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:01.586915 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-a4ab5867-9b59-4994-a19a-79b61ae5eb1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6eb1f3996b34589b57f3ac0730e08ab'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:01.587630 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-a4ab5867-9b59-4994-a19a-79b61ae5eb1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6eb1f3996b34589b57f3ac0730e08ab'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:01.593575 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1391/2785] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 17:28:01 2022] DELETE /identity/v3/users/a6eb1f3996b34589b57f3ac0730e08ab => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:28:01.992296 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37afd678-8219-4de2-ac76-e217ae9c2c46 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:28:01.994484 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-37afd678-8219-4de2-ac76-e217ae9c2c46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:28:02.008807 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37afd678-8219-4de2-ac76-e217ae9c2c46 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:28:02.067033 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45fb22f-0e5f-4120-a394-fa8c5a100014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:28:02.067871 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45fb22f-0e5f-4120-a394-fa8c5a100014 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:02.068266 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45fb22f-0e5f-4120-a394-fa8c5a100014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:02.068661 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45fb22f-0e5f-4120-a394-fa8c5a100014 admin admin] PATH_INFO: `/v3/projects/e3d1195d7adb440eabb981c535dbb282` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:02.069868 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45fb22f-0e5f-4120-a394-fa8c5a100014 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3d1195d7adb440eabb981c535dbb282)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:28:02.071922 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45fb22f-0e5f-4120-a394-fa8c5a100014 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:28:02.166146 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:28:02.178538 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-b45fb22f-0e5f-4120-a394-fa8c5a100014 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3d1195d7adb440eabb981c535dbb282', 'request_id': 'req-b45fb22f-0e5f-4120-a394-fa8c5a100014'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:02.185272 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-b45fb22f-0e5f-4120-a394-fa8c5a100014 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3d1195d7adb440eabb981c535dbb282', 'request_id': 'req-b45fb22f-0e5f-4120-a394-fa8c5a100014'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:02.256674 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-b45fb22f-0e5f-4120-a394-fa8c5a100014 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3d1195d7adb440eabb981c535dbb282', 'request_id': 'req-b45fb22f-0e5f-4120-a394-fa8c5a100014'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:02.267473 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-b45fb22f-0e5f-4120-a394-fa8c5a100014 admin admin] The token cache is being invalidate because project e3d1195d7adb440eabb981c535dbb282 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 17:28:02.268717 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-b45fb22f-0e5f-4120-a394-fa8c5a100014 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:02.269784 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-b45fb22f-0e5f-4120-a394-fa8c5a100014 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:02.271800 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1395/2786] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 17:28:01 2022] DELETE /identity/v3/projects/e3d1195d7adb440eabb981c535dbb282 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:28:02.276028 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ab5867-9b59-4994-a19a-79b61ae5eb1d admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:28:02.277470 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a4ab5867-9b59-4994-a19a-79b61ae5eb1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:28:02.290191 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ab5867-9b59-4994-a19a-79b61ae5eb1d admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:28:02.349366 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bca81d-a182-4f2c-8dff-4d67f055c172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:28:02.350157 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bca81d-a182-4f2c-8dff-4d67f055c172 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:02.350437 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bca81d-a182-4f2c-8dff-4d67f055c172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:02.350710 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bca81d-a182-4f2c-8dff-4d67f055c172 admin admin] PATH_INFO: `/v3/users/612ecf415f994674895e2a404cdbd318` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:02.351928 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bca81d-a182-4f2c-8dff-4d67f055c172 admin admin] RBAC: Authorizing `identity:delete_user(user_id=612ecf415f994674895e2a404cdbd318)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:28:02.353857 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bca81d-a182-4f2c-8dff-4d67f055c172 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:28:02.501243 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-36bca81d-a182-4f2c-8dff-4d67f055c172 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '612ecf415f994674895e2a404cdbd318'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:02.543822 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-36bca81d-a182-4f2c-8dff-4d67f055c172 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '612ecf415f994674895e2a404cdbd318'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:02.548733 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:28:02.552319 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-36bca81d-a182-4f2c-8dff-4d67f055c172 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '612ecf415f994674895e2a404cdbd318'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:02.565368 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-36bca81d-a182-4f2c-8dff-4d67f055c172 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '612ecf415f994674895e2a404cdbd318'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:02.566032 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-36bca81d-a182-4f2c-8dff-4d67f055c172 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '612ecf415f994674895e2a404cdbd318'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:02.571963 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1392/2787] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 17:28:02 2022] DELETE /identity/v3/users/612ecf415f994674895e2a404cdbd318 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:28:02.962704 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45fb22f-0e5f-4120-a394-fa8c5a100014 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:28:02.964289 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-b45fb22f-0e5f-4120-a394-fa8c5a100014 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:28:02.978496 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45fb22f-0e5f-4120-a394-fa8c5a100014 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:28:03.036809 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f50bb2-33b2-47d3-b171-20e9ed08b63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:28:03.037685 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f50bb2-33b2-47d3-b171-20e9ed08b63f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:03.037970 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f50bb2-33b2-47d3-b171-20e9ed08b63f admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:03.038250 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f50bb2-33b2-47d3-b171-20e9ed08b63f admin admin] PATH_INFO: `/v3/projects/01ac0e5c57444334a3215c2cdf35b16c` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:03.039442 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f50bb2-33b2-47d3-b171-20e9ed08b63f admin admin] RBAC: Authorizing `identity:delete_project(project_id=01ac0e5c57444334a3215c2cdf35b16c)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:28:03.041376 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f50bb2-33b2-47d3-b171-20e9ed08b63f admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:28:03.156601 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-e7f50bb2-33b2-47d3-b171-20e9ed08b63f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01ac0e5c57444334a3215c2cdf35b16c', 'request_id': 'req-e7f50bb2-33b2-47d3-b171-20e9ed08b63f'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:03.162993 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-e7f50bb2-33b2-47d3-b171-20e9ed08b63f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01ac0e5c57444334a3215c2cdf35b16c', 'request_id': 'req-e7f50bb2-33b2-47d3-b171-20e9ed08b63f'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:03.229584 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-e7f50bb2-33b2-47d3-b171-20e9ed08b63f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01ac0e5c57444334a3215c2cdf35b16c', 'request_id': 'req-e7f50bb2-33b2-47d3-b171-20e9ed08b63f'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:03.240739 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-e7f50bb2-33b2-47d3-b171-20e9ed08b63f admin admin] The token cache is being invalidate because project 01ac0e5c57444334a3215c2cdf35b16c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 17:28:03.241050 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-e7f50bb2-33b2-47d3-b171-20e9ed08b63f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:03.248224 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-e7f50bb2-33b2-47d3-b171-20e9ed08b63f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:03.248224 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1396/2788] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 17:28:02 2022] DELETE /identity/v3/projects/01ac0e5c57444334a3215c2cdf35b16c => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:28:03.249239 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b08b4b5-e78f-4cce-9365-659a2ec6a7bf None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:03.249525 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b08b4b5-e78f-4cce-9365-659a2ec6a7bf None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:03.249800 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b08b4b5-e78f-4cce-9365-659a2ec6a7bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:03.268997 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-6b08b4b5-e78f-4cce-9365-659a2ec6a7bf None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:28:03.270201 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-6b08b4b5-e78f-4cce-9365-659a2ec6a7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:28:03.328620 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1393/2789] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:28:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 17:28:03.337027 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3abee-51d9-4359-9a81-8bfdb9df6993 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:03.337320 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3abee-51d9-4359-9a81-8bfdb9df6993 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:03.337600 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3abee-51d9-4359-9a81-8bfdb9df6993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:03.356894 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-cad3abee-51d9-4359-9a81-8bfdb9df6993 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:28:03.358093 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-cad3abee-51d9-4359-9a81-8bfdb9df6993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:28:03.366646 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1397/2790] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:28:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:28:03.370712 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b08b4b5-e78f-4cce-9365-659a2ec6a7bf None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:28:03.381042 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b08b4b5-e78f-4cce-9365-659a2ec6a7bf None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:28:03.385792 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a4a3f-5631-49e6-a899-2bb3e4537f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:28:03.386444 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a4a3f-5631-49e6-a899-2bb3e4537f28 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:03.386726 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a4a3f-5631-49e6-a899-2bb3e4537f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:03.387002 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a4a3f-5631-49e6-a899-2bb3e4537f28 admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:03.387760 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a4a3f-5631-49e6-a899-2bb3e4537f28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 17:28:03.388067 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a4a3f-5631-49e6-a899-2bb3e4537f28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:28:03.389673 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a4a3f-5631-49e6-a899-2bb3e4537f28 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:28:03.401105 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1394/2791] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 17:28:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 17:28:03.405302 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad3abee-51d9-4359-9a81-8bfdb9df6993 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:28:03.414750 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad3abee-51d9-4359-9a81-8bfdb9df6993 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:28:03.419456 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cbfee2-ce13-4bb8-a4a7-a14cfc004f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:28:03.420132 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cbfee2-ce13-4bb8-a4a7-a14cfc004f1c admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:03.420417 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cbfee2-ce13-4bb8-a4a7-a14cfc004f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:03.420692 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cbfee2-ce13-4bb8-a4a7-a14cfc004f1c admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:03.421390 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cbfee2-ce13-4bb8-a4a7-a14cfc004f1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:28:03.423269 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cbfee2-ce13-4bb8-a4a7-a14cfc004f1c admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:28:03.510228 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1398/2792] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 17:28:03 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:28:03.514523 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a4a3f-5631-49e6-a899-2bb3e4537f28 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:28:03.524752 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a4a3f-5631-49e6-a899-2bb3e4537f28 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:28:03.581156 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66046170-313c-4c70-bef2-3a9418fe71b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:28:03.581822 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66046170-313c-4c70-bef2-3a9418fe71b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:03.582110 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66046170-313c-4c70-bef2-3a9418fe71b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:03.582391 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66046170-313c-4c70-bef2-3a9418fe71b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:03.583132 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66046170-313c-4c70-bef2-3a9418fe71b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:28:03.585089 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66046170-313c-4c70-bef2-3a9418fe71b2 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:28:03.657507 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1395/2793] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 17:28:03 2022] POST /identity/v3/users => generated 293 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:28:03.662496 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cbfee2-ce13-4bb8-a4a7-a14cfc004f1c admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:28:03.672821 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cbfee2-ce13-4bb8-a4a7-a14cfc004f1c admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:28:03.678320 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c3765e-c525-4f5f-bf34-867e62fba607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:28:03.679023 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3765e-c525-4f5f-bf34-867e62fba607 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:03.679392 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3765e-c525-4f5f-bf34-867e62fba607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:03.679716 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3765e-c525-4f5f-bf34-867e62fba607 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:03.680454 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c3765e-c525-4f5f-bf34-867e62fba607 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:28:03.682392 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c3765e-c525-4f5f-bf34-867e62fba607 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:28:03.693181 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1399/2794] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 17:28:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:28:03.699380 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66046170-313c-4c70-bef2-3a9418fe71b2 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:28:03.709514 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66046170-313c-4c70-bef2-3a9418fe71b2 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:28:03.714377 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c5d882-b82b-4f67-8b53-ee62aa1da860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:28:03.715032 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5d882-b82b-4f67-8b53-ee62aa1da860 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:03.715319 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5d882-b82b-4f67-8b53-ee62aa1da860 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:03.715659 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5d882-b82b-4f67-8b53-ee62aa1da860 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:03.716530 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c5d882-b82b-4f67-8b53-ee62aa1da860 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:28:03.718170 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c5d882-b82b-4f67-8b53-ee62aa1da860 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:28:03.730373 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1396/2795] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 17:28:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:28:03.734387 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c3765e-c525-4f5f-bf34-867e62fba607 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:28:03.745039 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c3765e-c525-4f5f-bf34-867e62fba607 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:28:03.749648 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62ff96b-0cc5-4681-83a9-87f81e948a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:28:03.750518 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ff96b-0cc5-4681-83a9-87f81e948a5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:03.750799 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ff96b-0cc5-4681-83a9-87f81e948a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:03.751073 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ff96b-0cc5-4681-83a9-87f81e948a5b admin admin] PATH_INFO: `/v3/projects/c5feda66d67e455ab00371352f003edf/users/f78457b99a9a46a39a5fa70a2c06e0ec/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:03.759035 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:28:03.770367 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62ff96b-0cc5-4681-83a9-87f81e948a5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5feda66d67e455ab00371352f003edf, user_id=f78457b99a9a46a39a5fa70a2c06e0ec, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:28:03.770367 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62ff96b-0cc5-4681-83a9-87f81e948a5b admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:28:03.858445 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1400/2796] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 17:28:03 2022] PUT /identity/v3/projects/c5feda66d67e455ab00371352f003edf/users/f78457b99a9a46a39a5fa70a2c06e0ec/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:28:03.871886 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494bd80-fc15-4c97-9a26-cf4fdece2d24 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:03.872178 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494bd80-fc15-4c97-9a26-cf4fdece2d24 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:03.872458 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494bd80-fc15-4c97-9a26-cf4fdece2d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:03.900991 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-4494bd80-fc15-4c97-9a26-cf4fdece2d24 None None] MFA Rules not processed for user `f78457b99a9a46a39a5fa70a2c06e0ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:28:03.902237 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-4494bd80-fc15-4c97-9a26-cf4fdece2d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:28:03.932417 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:28:03.966649 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1397/2797] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:28:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:28:03.971778 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62ff96b-0cc5-4681-83a9-87f81e948a5b admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:28:04.034387 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62ff96b-0cc5-4681-83a9-87f81e948a5b admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:28:04.038984 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c2f224-38e5-40f3-9f6c-6f9153d7adb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:28:04.039679 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c2f224-38e5-40f3-9f6c-6f9153d7adb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:04.039969 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c2f224-38e5-40f3-9f6c-6f9153d7adb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:04.040247 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c2f224-38e5-40f3-9f6c-6f9153d7adb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:04.040946 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c2f224-38e5-40f3-9f6c-6f9153d7adb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:28:04.042843 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c2f224-38e5-40f3-9f6c-6f9153d7adb8 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:28:04.145823 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1401/2798] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 17:28:03 2022] POST /identity/v3/projects => generated 385 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:28:04.149974 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4494bd80-fc15-4c97-9a26-cf4fdece2d24 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:28:04.161529 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4494bd80-fc15-4c97-9a26-cf4fdece2d24 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:28:04.216547 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7daebf-adc3-4789-a5b9-6391dc66c69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:28:04.217210 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7daebf-adc3-4789-a5b9-6391dc66c69c admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:04.217495 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7daebf-adc3-4789-a5b9-6391dc66c69c admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:04.217771 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7daebf-adc3-4789-a5b9-6391dc66c69c admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:04.218459 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7daebf-adc3-4789-a5b9-6391dc66c69c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:28:04.220434 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7daebf-adc3-4789-a5b9-6391dc66c69c admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:28:04.315232 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1398/2799] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 17:28:04 2022] POST /identity/v3/users => generated 299 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:28:04.319027 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c2f224-38e5-40f3-9f6c-6f9153d7adb8 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:28:04.329654 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c2f224-38e5-40f3-9f6c-6f9153d7adb8 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:28:04.335023 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3d5168-25b6-4109-b271-0b25fd2405d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:28:04.335727 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d5168-25b6-4109-b271-0b25fd2405d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:04.336015 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d5168-25b6-4109-b271-0b25fd2405d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:04.336327 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d5168-25b6-4109-b271-0b25fd2405d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:04.337053 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3d5168-25b6-4109-b271-0b25fd2405d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:28:04.338846 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3d5168-25b6-4109-b271-0b25fd2405d5 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:28:04.350734 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1402/2800] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 17:28:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:28:04.355562 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7daebf-adc3-4789-a5b9-6391dc66c69c admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:28:04.365601 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7daebf-adc3-4789-a5b9-6391dc66c69c admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:28:04.370150 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c193ffd-0206-40d5-b0de-5f9dc268a532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:28:04.371016 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c193ffd-0206-40d5-b0de-5f9dc268a532 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:04.371297 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c193ffd-0206-40d5-b0de-5f9dc268a532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:04.371612 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c193ffd-0206-40d5-b0de-5f9dc268a532 admin admin] PATH_INFO: `/v3/projects/4888fcaf279e42ec8f7b929413204b33/users/e0caba318b8e467c876939510903a49a/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:04.386952 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c193ffd-0206-40d5-b0de-5f9dc268a532 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4888fcaf279e42ec8f7b929413204b33, user_id=e0caba318b8e467c876939510903a49a, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:28:04.389630 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c193ffd-0206-40d5-b0de-5f9dc268a532 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:28:04.492114 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1399/2801] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 17:28:04 2022] PUT /identity/v3/projects/4888fcaf279e42ec8f7b929413204b33/users/e0caba318b8e467c876939510903a49a/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:28:04.496093 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3d5168-25b6-4109-b271-0b25fd2405d5 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:28:04.506863 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3d5168-25b6-4109-b271-0b25fd2405d5 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:28:04.563893 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1513e6-a69a-4a22-92a2-603cb681ea45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:28:04.565094 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1513e6-a69a-4a22-92a2-603cb681ea45 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:04.565433 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1513e6-a69a-4a22-92a2-603cb681ea45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:04.565658 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1513e6-a69a-4a22-92a2-603cb681ea45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:04.566383 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1513e6-a69a-4a22-92a2-603cb681ea45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:28:04.568010 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1513e6-a69a-4a22-92a2-603cb681ea45 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:28:04.577599 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1403/2802] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 17:28:04 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:28:04.582559 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c193ffd-0206-40d5-b0de-5f9dc268a532 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:28:04.592941 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c193ffd-0206-40d5-b0de-5f9dc268a532 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:28:04.597630 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b322ff-0444-4f22-9949-fa431d1d0534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:28:04.598503 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b322ff-0444-4f22-9949-fa431d1d0534 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:04.598789 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b322ff-0444-4f22-9949-fa431d1d0534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:04.599066 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b322ff-0444-4f22-9949-fa431d1d0534 admin admin] PATH_INFO: `/v3/projects/4888fcaf279e42ec8f7b929413204b33/users/e0caba318b8e467c876939510903a49a/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:04.601049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b322ff-0444-4f22-9949-fa431d1d0534 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4888fcaf279e42ec8f7b929413204b33, user_id=e0caba318b8e467c876939510903a49a, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:28:04.603709 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b322ff-0444-4f22-9949-fa431d1d0534 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:28:04.613860 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1400/2803] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 17:28:04 2022] PUT /identity/v3/projects/4888fcaf279e42ec8f7b929413204b33/users/e0caba318b8e467c876939510903a49a/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:28:04.624850 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67430b7-3260-4ab0-8b12-a8db6bb032dd None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:04.625618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67430b7-3260-4ab0-8b12-a8db6bb032dd None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:04.625618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67430b7-3260-4ab0-8b12-a8db6bb032dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:04.652008 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-e67430b7-3260-4ab0-8b12-a8db6bb032dd None None] MFA Rules not processed for user `e0caba318b8e467c876939510903a49a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:28:04.653207 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-e67430b7-3260-4ab0-8b12-a8db6bb032dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:28:04.697282 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:28:04.720304 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1404/2804] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:28:04 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:28:04.731792 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b322ff-0444-4f22-9949-fa431d1d0534 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:28:04.733487 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-73b322ff-0444-4f22-9949-fa431d1d0534 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:28:04.790327 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b322ff-0444-4f22-9949-fa431d1d0534 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:28:04.793966 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03a164c-6e23-4172-ac40-a90826005292 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:28:04.794606 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a164c-6e23-4172-ac40-a90826005292 service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:04.794891 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a164c-6e23-4172-ac40-a90826005292 service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:04.795198 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a164c-6e23-4172-ac40-a90826005292 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:04.804775 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03a164c-6e23-4172-ac40-a90826005292 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:28:04.806460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03a164c-6e23-4172-ac40-a90826005292 service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:28:04.816405 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1401/2805] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:28:04 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:28:05.774997 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67430b7-3260-4ab0-8b12-a8db6bb032dd None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:28:05.841854 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67430b7-3260-4ab0-8b12-a8db6bb032dd None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:28:05.846367 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bbfd3d-232a-4322-8df1-5f419c3b4e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:28:05.847250 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bbfd3d-232a-4322-8df1-5f419c3b4e54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:05.847562 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bbfd3d-232a-4322-8df1-5f419c3b4e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:05.847847 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bbfd3d-232a-4322-8df1-5f419c3b4e54 admin admin] PATH_INFO: `/v3/users/f78457b99a9a46a39a5fa70a2c06e0ec` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:05.849047 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bbfd3d-232a-4322-8df1-5f419c3b4e54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f78457b99a9a46a39a5fa70a2c06e0ec)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:28:05.850970 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bbfd3d-232a-4322-8df1-5f419c3b4e54 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:28:06.054029 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-66bbfd3d-232a-4322-8df1-5f419c3b4e54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f78457b99a9a46a39a5fa70a2c06e0ec'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:06.097959 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-66bbfd3d-232a-4322-8df1-5f419c3b4e54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f78457b99a9a46a39a5fa70a2c06e0ec'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:06.098607 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-66bbfd3d-232a-4322-8df1-5f419c3b4e54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f78457b99a9a46a39a5fa70a2c06e0ec'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:06.099283 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-66bbfd3d-232a-4322-8df1-5f419c3b4e54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f78457b99a9a46a39a5fa70a2c06e0ec'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:06.106796 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-66bbfd3d-232a-4322-8df1-5f419c3b4e54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f78457b99a9a46a39a5fa70a2c06e0ec'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:06.124620 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1405/2806] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 17:28:05 2022] DELETE /identity/v3/users/f78457b99a9a46a39a5fa70a2c06e0ec => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:28:06.136790 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5016a974-e416-4305-9f9f-70a97a27d0b5 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:06.137286 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5016a974-e416-4305-9f9f-70a97a27d0b5 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:06.137391 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5016a974-e416-4305-9f9f-70a97a27d0b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:06.155536 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-5016a974-e416-4305-9f9f-70a97a27d0b5 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:28:06.156707 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-5016a974-e416-4305-9f9f-70a97a27d0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:28:06.176782 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:28:06.215226 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1402/2807] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:28:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 17:28:06.225143 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bbfd3d-232a-4322-8df1-5f419c3b4e54 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:28:06.226608 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-66bbfd3d-232a-4322-8df1-5f419c3b4e54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:28:06.283859 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bbfd3d-232a-4322-8df1-5f419c3b4e54 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:28:06.287675 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d65535-ffcc-42f1-9f3f-cbd2f2c32735 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:28:06.288326 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d65535-ffcc-42f1-9f3f-cbd2f2c32735 service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:06.288610 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d65535-ffcc-42f1-9f3f-cbd2f2c32735 service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:06.288886 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d65535-ffcc-42f1-9f3f-cbd2f2c32735 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:06.298015 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d65535-ffcc-42f1-9f3f-cbd2f2c32735 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:28:06.300001 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d65535-ffcc-42f1-9f3f-cbd2f2c32735 service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:28:06.309828 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1406/2808] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:28:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:28:06.722653 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5016a974-e416-4305-9f9f-70a97a27d0b5 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:28:06.724117 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-5016a974-e416-4305-9f9f-70a97a27d0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:28:06.738416 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5016a974-e416-4305-9f9f-70a97a27d0b5 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:28:06.743023 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dfa341-bff2-47de-b88d-87583e4e2041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:28:06.744585 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dfa341-bff2-47de-b88d-87583e4e2041 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:06.749685 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dfa341-bff2-47de-b88d-87583e4e2041 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:06.749685 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dfa341-bff2-47de-b88d-87583e4e2041 admin admin] PATH_INFO: `/v3/projects/c5feda66d67e455ab00371352f003edf` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:06.749685 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dfa341-bff2-47de-b88d-87583e4e2041 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5feda66d67e455ab00371352f003edf)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:28:06.749685 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dfa341-bff2-47de-b88d-87583e4e2041 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:28:06.865105 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-76dfa341-bff2-47de-b88d-87583e4e2041 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5feda66d67e455ab00371352f003edf', 'request_id': 'req-76dfa341-bff2-47de-b88d-87583e4e2041'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:06.870649 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-76dfa341-bff2-47de-b88d-87583e4e2041 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5feda66d67e455ab00371352f003edf', 'request_id': 'req-76dfa341-bff2-47de-b88d-87583e4e2041'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:06.876816 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-76dfa341-bff2-47de-b88d-87583e4e2041 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5feda66d67e455ab00371352f003edf', 'request_id': 'req-76dfa341-bff2-47de-b88d-87583e4e2041'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:06.959545 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-76dfa341-bff2-47de-b88d-87583e4e2041 admin admin] The token cache is being invalidate because project c5feda66d67e455ab00371352f003edf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 17:28:06.959857 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-76dfa341-bff2-47de-b88d-87583e4e2041 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:06.960494 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-76dfa341-bff2-47de-b88d-87583e4e2041 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:06.962049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1403/2809] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 17:28:06 2022] DELETE /identity/v3/projects/c5feda66d67e455ab00371352f003edf => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:28:06.966127 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d65535-ffcc-42f1-9f3f-cbd2f2c32735 service neutron] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:28:06.972819 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:28:06.976511 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d65535-ffcc-42f1-9f3f-cbd2f2c32735 service neutron] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:28:07.032690 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f898d6f1-4040-41b3-b03d-64b0c852db2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:28:07.034277 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f898d6f1-4040-41b3-b03d-64b0c852db2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:07.034553 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f898d6f1-4040-41b3-b03d-64b0c852db2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:07.034826 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f898d6f1-4040-41b3-b03d-64b0c852db2c admin admin] PATH_INFO: `/v3/users/e0caba318b8e467c876939510903a49a` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:07.036054 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f898d6f1-4040-41b3-b03d-64b0c852db2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0caba318b8e467c876939510903a49a)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:28:07.037961 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f898d6f1-4040-41b3-b03d-64b0c852db2c admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:28:07.190992 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-f898d6f1-4040-41b3-b03d-64b0c852db2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0caba318b8e467c876939510903a49a'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:07.257787 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-f898d6f1-4040-41b3-b03d-64b0c852db2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0caba318b8e467c876939510903a49a'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:07.258424 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-f898d6f1-4040-41b3-b03d-64b0c852db2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0caba318b8e467c876939510903a49a'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:07.259057 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-f898d6f1-4040-41b3-b03d-64b0c852db2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0caba318b8e467c876939510903a49a'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:07.266501 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-f898d6f1-4040-41b3-b03d-64b0c852db2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0caba318b8e467c876939510903a49a'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:07.285903 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1407/2810] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 17:28:06 2022] DELETE /identity/v3/users/e0caba318b8e467c876939510903a49a => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:28:07.636760 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dfa341-bff2-47de-b88d-87583e4e2041 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:28:07.638207 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-76dfa341-bff2-47de-b88d-87583e4e2041 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:28:07.652136 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dfa341-bff2-47de-b88d-87583e4e2041 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:28:07.709172 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7458c1-bddd-4b8d-8891-c58afc36d764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:28:07.710125 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7458c1-bddd-4b8d-8891-c58afc36d764 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:07.710409 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7458c1-bddd-4b8d-8891-c58afc36d764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:07.710681 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7458c1-bddd-4b8d-8891-c58afc36d764 admin admin] PATH_INFO: `/v3/projects/4888fcaf279e42ec8f7b929413204b33` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:07.711980 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7458c1-bddd-4b8d-8891-c58afc36d764 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4888fcaf279e42ec8f7b929413204b33)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:28:07.714116 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7458c1-bddd-4b8d-8891-c58afc36d764 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:28:07.794829 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-4b7458c1-bddd-4b8d-8891-c58afc36d764 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4888fcaf279e42ec8f7b929413204b33', 'request_id': 'req-4b7458c1-bddd-4b8d-8891-c58afc36d764'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:07.796858 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:28:07.801277 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-4b7458c1-bddd-4b8d-8891-c58afc36d764 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4888fcaf279e42ec8f7b929413204b33', 'request_id': 'req-4b7458c1-bddd-4b8d-8891-c58afc36d764'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:07.807707 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-4b7458c1-bddd-4b8d-8891-c58afc36d764 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4888fcaf279e42ec8f7b929413204b33', 'request_id': 'req-4b7458c1-bddd-4b8d-8891-c58afc36d764'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:07.856957 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-4b7458c1-bddd-4b8d-8891-c58afc36d764 admin admin] The token cache is being invalidate because project 4888fcaf279e42ec8f7b929413204b33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 17:28:07.857253 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-4b7458c1-bddd-4b8d-8891-c58afc36d764 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:07.857884 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-4b7458c1-bddd-4b8d-8891-c58afc36d764 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:07.859433 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1404/2811] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 17:28:07 2022] DELETE /identity/v3/projects/4888fcaf279e42ec8f7b929413204b33 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:28:07.864600 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e99647-97aa-44eb-88ea-211004b0b965 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:07.864895 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e99647-97aa-44eb-88ea-211004b0b965 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:07.865181 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e99647-97aa-44eb-88ea-211004b0b965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:07.883064 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-a9e99647-97aa-44eb-88ea-211004b0b965 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:28:07.884310 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-a9e99647-97aa-44eb-88ea-211004b0b965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:28:07.944214 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1408/2812] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:28:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:28:07.952013 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad7684e-2dff-4e0f-8918-666eb0355191 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:07.952303 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad7684e-2dff-4e0f-8918-666eb0355191 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:07.952580 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad7684e-2dff-4e0f-8918-666eb0355191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:07.970956 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-aad7684e-2dff-4e0f-8918-666eb0355191 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:28:07.972126 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-aad7684e-2dff-4e0f-8918-666eb0355191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:28:07.979820 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1405/2813] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:28:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:28:07.983842 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e99647-97aa-44eb-88ea-211004b0b965 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:28:07.993749 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e99647-97aa-44eb-88ea-211004b0b965 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:28:07.998210 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab169124-ab5c-4cc0-9618-51b4e4af91fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:28:07.998858 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab169124-ab5c-4cc0-9618-51b4e4af91fe admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:07.999135 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab169124-ab5c-4cc0-9618-51b4e4af91fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:07.999406 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab169124-ab5c-4cc0-9618-51b4e4af91fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:08.000155 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab169124-ab5c-4cc0-9618-51b4e4af91fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 17:28:08.000452 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab169124-ab5c-4cc0-9618-51b4e4af91fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:28:08.002585 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab169124-ab5c-4cc0-9618-51b4e4af91fe admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:28:08.013874 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1409/2814] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 17:28:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 17:28:08.018160 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad7684e-2dff-4e0f-8918-666eb0355191 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:28:08.027682 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad7684e-2dff-4e0f-8918-666eb0355191 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:28:08.032377 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a2a02-ba9c-4c1d-ba68-bc953f284d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:28:08.033588 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a2a02-ba9c-4c1d-ba68-bc953f284d61 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:08.033956 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a2a02-ba9c-4c1d-ba68-bc953f284d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:08.034246 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a2a02-ba9c-4c1d-ba68-bc953f284d61 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:08.034945 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539a2a02-ba9c-4c1d-ba68-bc953f284d61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:28:08.036945 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539a2a02-ba9c-4c1d-ba68-bc953f284d61 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:28:08.121964 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1406/2815] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 17:28:08 2022] POST /identity/v3/projects => generated 359 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:28:08.126194 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab169124-ab5c-4cc0-9618-51b4e4af91fe admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:28:08.136469 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab169124-ab5c-4cc0-9618-51b4e4af91fe admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:28:08.144317 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:28:08.193668 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0919cca7-75cb-4715-a52f-bdbc59c4c018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:28:08.194462 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919cca7-75cb-4715-a52f-bdbc59c4c018 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:08.194884 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919cca7-75cb-4715-a52f-bdbc59c4c018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:08.195274 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919cca7-75cb-4715-a52f-bdbc59c4c018 admin admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:08.196141 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0919cca7-75cb-4715-a52f-bdbc59c4c018 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:28:08.198163 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0919cca7-75cb-4715-a52f-bdbc59c4c018 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:28:08.275581 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1410/2816] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 17:28:08 2022] POST /identity/v3/users => generated 280 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:28:08.279816 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a2a02-ba9c-4c1d-ba68-bc953f284d61 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:28:08.290769 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a2a02-ba9c-4c1d-ba68-bc953f284d61 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:28:08.295423 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ef57a-c3e0-47bb-954c-3ba6cf72a9c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:28:08.296113 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ef57a-c3e0-47bb-954c-3ba6cf72a9c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:08.296401 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ef57a-c3e0-47bb-954c-3ba6cf72a9c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:08.296683 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ef57a-c3e0-47bb-954c-3ba6cf72a9c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:08.297411 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98ef57a-c3e0-47bb-954c-3ba6cf72a9c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:28:08.299693 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98ef57a-c3e0-47bb-954c-3ba6cf72a9c1 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:28:08.311305 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1407/2817] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 17:28:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:28:08.315204 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0919cca7-75cb-4715-a52f-bdbc59c4c018 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:28:08.325281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0919cca7-75cb-4715-a52f-bdbc59c4c018 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:28:08.329913 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8e035f-66f8-4cb6-919d-8cf61844f6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:28:08.330552 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e035f-66f8-4cb6-919d-8cf61844f6f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:08.330835 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e035f-66f8-4cb6-919d-8cf61844f6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:08.331110 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e035f-66f8-4cb6-919d-8cf61844f6f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:08.331879 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8e035f-66f8-4cb6-919d-8cf61844f6f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:28:08.333473 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8e035f-66f8-4cb6-919d-8cf61844f6f9 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:28:08.343269 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1411/2818] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 17:28:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:28:08.347846 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ef57a-c3e0-47bb-954c-3ba6cf72a9c1 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:28:08.357597 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ef57a-c3e0-47bb-954c-3ba6cf72a9c1 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:28:08.362915 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852c5cbb-0326-45e6-823d-d3a9b355249a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:28:08.363811 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c5cbb-0326-45e6-823d-d3a9b355249a admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:08.364091 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c5cbb-0326-45e6-823d-d3a9b355249a admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:08.364365 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c5cbb-0326-45e6-823d-d3a9b355249a admin admin] PATH_INFO: `/v3/projects/fcee21720c414d42bc0e3273429f3525/users/cd17ef5c7ab84ea190f5d91338bddf21/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:08.378291 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852c5cbb-0326-45e6-823d-d3a9b355249a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcee21720c414d42bc0e3273429f3525, user_id=cd17ef5c7ab84ea190f5d91338bddf21, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:28:08.380939 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852c5cbb-0326-45e6-823d-d3a9b355249a admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:28:08.446511 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1408/2819] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 17:28:08 2022] PUT /identity/v3/projects/fcee21720c414d42bc0e3273429f3525/users/cd17ef5c7ab84ea190f5d91338bddf21/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:28:08.457582 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44afb145-f6f8-41e5-a9ad-67f10b2dc164 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:08.457878 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44afb145-f6f8-41e5-a9ad-67f10b2dc164 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:08.458161 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44afb145-f6f8-41e5-a9ad-67f10b2dc164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:08.484917 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-44afb145-f6f8-41e5-a9ad-67f10b2dc164 None None] MFA Rules not processed for user `cd17ef5c7ab84ea190f5d91338bddf21`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:28:08.486165 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-44afb145-f6f8-41e5-a9ad-67f10b2dc164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:28:08.549294 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1412/2820] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:28:08 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:28:08.587559 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852c5cbb-0326-45e6-823d-d3a9b355249a admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:28:08.589216 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-852c5cbb-0326-45e6-823d-d3a9b355249a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:28:08.645066 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852c5cbb-0326-45e6-823d-d3a9b355249a admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:28:08.648651 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121bb3ed-0c56-4f67-9138-2ec0fd58e957 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:28:08.649286 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121bb3ed-0c56-4f67-9138-2ec0fd58e957 service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:08.649564 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121bb3ed-0c56-4f67-9138-2ec0fd58e957 service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:08.649835 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121bb3ed-0c56-4f67-9138-2ec0fd58e957 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:08.657849 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121bb3ed-0c56-4f67-9138-2ec0fd58e957 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:28:08.660166 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121bb3ed-0c56-4f67-9138-2ec0fd58e957 service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:28:08.668665 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1409/2821] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:28:08 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:28:08.682489 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44afb145-f6f8-41e5-a9ad-67f10b2dc164 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:28:08.744078 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44afb145-f6f8-41e5-a9ad-67f10b2dc164 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:28:08.748636 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaf1327-9f20-48af-98b2-fe2fea1c854a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:28:08.749510 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaf1327-9f20-48af-98b2-fe2fea1c854a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:08.749790 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaf1327-9f20-48af-98b2-fe2fea1c854a admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:08.750064 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaf1327-9f20-48af-98b2-fe2fea1c854a admin admin] PATH_INFO: `/v3/users/cd17ef5c7ab84ea190f5d91338bddf21` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:08.751257 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaf1327-9f20-48af-98b2-fe2fea1c854a admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd17ef5c7ab84ea190f5d91338bddf21)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:28:08.753984 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaf1327-9f20-48af-98b2-fe2fea1c854a admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:28:08.775458 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:28:08.968501 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-beaf1327-9f20-48af-98b2-fe2fea1c854a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd17ef5c7ab84ea190f5d91338bddf21'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:09.040739 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-beaf1327-9f20-48af-98b2-fe2fea1c854a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd17ef5c7ab84ea190f5d91338bddf21'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:09.041346 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-beaf1327-9f20-48af-98b2-fe2fea1c854a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd17ef5c7ab84ea190f5d91338bddf21'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:09.041952 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-beaf1327-9f20-48af-98b2-fe2fea1c854a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd17ef5c7ab84ea190f5d91338bddf21'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:09.049680 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-beaf1327-9f20-48af-98b2-fe2fea1c854a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd17ef5c7ab84ea190f5d91338bddf21'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:09.067642 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1413/2822] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 17:28:08 2022] DELETE /identity/v3/users/cd17ef5c7ab84ea190f5d91338bddf21 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:28:09.072327 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f5fd4-e601-4b8f-9da5-a9d5ca8d9534 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:09.072611 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f5fd4-e601-4b8f-9da5-a9d5ca8d9534 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:09.073046 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f5fd4-e601-4b8f-9da5-a9d5ca8d9534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:09.090508 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-695f5fd4-e601-4b8f-9da5-a9d5ca8d9534 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:28:09.091657 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-695f5fd4-e601-4b8f-9da5-a9d5ca8d9534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:28:09.149779 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1410/2823] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:28:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 17:28:09.160219 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaf1327-9f20-48af-98b2-fe2fea1c854a admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:28:09.161631 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-beaf1327-9f20-48af-98b2-fe2fea1c854a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:28:09.226142 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaf1327-9f20-48af-98b2-fe2fea1c854a admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:28:09.230041 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0be7a16-0729-4086-bcca-24593d570321 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:28:09.230693 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0be7a16-0729-4086-bcca-24593d570321 service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:09.230982 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0be7a16-0729-4086-bcca-24593d570321 service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:09.231281 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0be7a16-0729-4086-bcca-24593d570321 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:09.241526 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0be7a16-0729-4086-bcca-24593d570321 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:28:09.242966 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0be7a16-0729-4086-bcca-24593d570321 service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:28:09.253453 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1414/2824] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:28:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:28:09.677116 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f5fd4-e601-4b8f-9da5-a9d5ca8d9534 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:28:09.678745 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-695f5fd4-e601-4b8f-9da5-a9d5ca8d9534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:28:09.693505 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f5fd4-e601-4b8f-9da5-a9d5ca8d9534 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:28:09.698407 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bd8df0-88c5-43f2-8122-e46504913af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:28:09.703959 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd8df0-88c5-43f2-8122-e46504913af2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:09.703959 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd8df0-88c5-43f2-8122-e46504913af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:09.703959 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd8df0-88c5-43f2-8122-e46504913af2 admin admin] PATH_INFO: `/v3/projects/fcee21720c414d42bc0e3273429f3525` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:09.703959 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bd8df0-88c5-43f2-8122-e46504913af2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcee21720c414d42bc0e3273429f3525)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:28:09.703959 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bd8df0-88c5-43f2-8122-e46504913af2 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:28:09.719952 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:28:09.807798 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-c5bd8df0-88c5-43f2-8122-e46504913af2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcee21720c414d42bc0e3273429f3525', 'request_id': 'req-c5bd8df0-88c5-43f2-8122-e46504913af2'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:09.813270 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-c5bd8df0-88c5-43f2-8122-e46504913af2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcee21720c414d42bc0e3273429f3525', 'request_id': 'req-c5bd8df0-88c5-43f2-8122-e46504913af2'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:09.820536 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-c5bd8df0-88c5-43f2-8122-e46504913af2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcee21720c414d42bc0e3273429f3525', 'request_id': 'req-c5bd8df0-88c5-43f2-8122-e46504913af2'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:09.920588 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-c5bd8df0-88c5-43f2-8122-e46504913af2 admin admin] The token cache is being invalidate because project fcee21720c414d42bc0e3273429f3525 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 17:28:09.920820 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-c5bd8df0-88c5-43f2-8122-e46504913af2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:09.921544 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-c5bd8df0-88c5-43f2-8122-e46504913af2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:09.923165 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1411/2825] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 17:28:09 2022] DELETE /identity/v3/projects/fcee21720c414d42bc0e3273429f3525 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:28:09.928677 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f865a9-249d-4b63-a7e6-501f435778c4 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:09.928967 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f865a9-249d-4b63-a7e6-501f435778c4 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:09.929249 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f865a9-249d-4b63-a7e6-501f435778c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:09.948195 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-51f865a9-249d-4b63-a7e6-501f435778c4 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:28:09.949443 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-51f865a9-249d-4b63-a7e6-501f435778c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:28:10.007430 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1415/2826] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:28:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:28:10.016208 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abba59da-1f9d-4e23-bae6-3b6a168122e9 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:10.016531 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abba59da-1f9d-4e23-bae6-3b6a168122e9 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:10.016816 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abba59da-1f9d-4e23-bae6-3b6a168122e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:10.036406 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-abba59da-1f9d-4e23-bae6-3b6a168122e9 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:28:10.037776 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-abba59da-1f9d-4e23-bae6-3b6a168122e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:28:10.045662 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1412/2827] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:28:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:28:10.057205 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f865a9-249d-4b63-a7e6-501f435778c4 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:28:10.066872 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f865a9-249d-4b63-a7e6-501f435778c4 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:28:10.071506 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e0e4f7-6f88-4134-a055-b6bf0cab2138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:28:10.072178 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0e4f7-6f88-4134-a055-b6bf0cab2138 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:10.072465 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0e4f7-6f88-4134-a055-b6bf0cab2138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:10.072740 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0e4f7-6f88-4134-a055-b6bf0cab2138 admin admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:10.073468 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e0e4f7-6f88-4134-a055-b6bf0cab2138 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 17:28:10.073774 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e0e4f7-6f88-4134-a055-b6bf0cab2138 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:28:10.075364 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e0e4f7-6f88-4134-a055-b6bf0cab2138 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:28:10.087450 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1416/2828] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 17:28:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 17:28:10.092200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abba59da-1f9d-4e23-bae6-3b6a168122e9 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:28:10.103099 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abba59da-1f9d-4e23-bae6-3b6a168122e9 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:28:10.108043 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729ce178-c01d-4582-8974-99a408c3b717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:28:10.108711 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ce178-c01d-4582-8974-99a408c3b717 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:10.108998 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ce178-c01d-4582-8974-99a408c3b717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:10.109275 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ce178-c01d-4582-8974-99a408c3b717 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:10.109989 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729ce178-c01d-4582-8974-99a408c3b717 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:28:10.112004 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729ce178-c01d-4582-8974-99a408c3b717 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:28:10.195278 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1413/2829] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 17:28:10 2022] POST /identity/v3/projects => generated 377 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:28:10.199351 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e0e4f7-6f88-4134-a055-b6bf0cab2138 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:28:10.210160 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e0e4f7-6f88-4134-a055-b6bf0cab2138 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:28:10.236280 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:28:10.268346 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70cef16-3ac1-4290-bd16-03c52daefd76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:28:10.269021 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70cef16-3ac1-4290-bd16-03c52daefd76 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:10.269307 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70cef16-3ac1-4290-bd16-03c52daefd76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:10.269587 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70cef16-3ac1-4290-bd16-03c52daefd76 admin admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:10.270288 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70cef16-3ac1-4290-bd16-03c52daefd76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:28:10.272231 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70cef16-3ac1-4290-bd16-03c52daefd76 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:28:10.371366 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1417/2830] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 17:28:10 2022] POST /identity/v3/users => generated 289 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:28:10.381224 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729ce178-c01d-4582-8974-99a408c3b717 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:28:10.386413 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729ce178-c01d-4582-8974-99a408c3b717 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:28:10.391016 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd6d6b-7080-476a-9936-e6d04ea750ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:28:10.391783 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd6d6b-7080-476a-9936-e6d04ea750ff admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:10.392069 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd6d6b-7080-476a-9936-e6d04ea750ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:10.392347 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd6d6b-7080-476a-9936-e6d04ea750ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:10.393986 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbd6d6b-7080-476a-9936-e6d04ea750ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:28:10.395605 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbd6d6b-7080-476a-9936-e6d04ea750ff admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:28:10.406648 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1414/2831] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 17:28:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:28:10.412827 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70cef16-3ac1-4290-bd16-03c52daefd76 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:28:10.423421 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70cef16-3ac1-4290-bd16-03c52daefd76 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:28:10.430126 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c98c44-4501-4a0e-b2d6-54b8bcd1a6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:28:10.430807 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c98c44-4501-4a0e-b2d6-54b8bcd1a6bf admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:10.431344 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c98c44-4501-4a0e-b2d6-54b8bcd1a6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:10.431686 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c98c44-4501-4a0e-b2d6-54b8bcd1a6bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:10.432428 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c98c44-4501-4a0e-b2d6-54b8bcd1a6bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:28:10.434096 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c98c44-4501-4a0e-b2d6-54b8bcd1a6bf admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:28:10.445464 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1418/2832] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 17:28:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:28:10.449865 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd6d6b-7080-476a-9936-e6d04ea750ff admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:28:10.460901 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd6d6b-7080-476a-9936-e6d04ea750ff admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:28:10.465554 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc219056-9c03-4146-8b96-7e46bb805467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:28:10.467300 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc219056-9c03-4146-8b96-7e46bb805467 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:10.467300 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc219056-9c03-4146-8b96-7e46bb805467 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:10.467300 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc219056-9c03-4146-8b96-7e46bb805467 admin admin] PATH_INFO: `/v3/projects/c9a2a6104d92436591f78a4d07c77592/users/28fc34fb729f48f9aa9cf76547f3d6eb/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:10.481687 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc219056-9c03-4146-8b96-7e46bb805467 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9a2a6104d92436591f78a4d07c77592, user_id=28fc34fb729f48f9aa9cf76547f3d6eb, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:28:10.484345 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc219056-9c03-4146-8b96-7e46bb805467 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:28:10.578645 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1415/2833] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 17:28:10 2022] PUT /identity/v3/projects/c9a2a6104d92436591f78a4d07c77592/users/28fc34fb729f48f9aa9cf76547f3d6eb/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:28:10.589605 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518d90e-88c4-476b-a6a7-7e75882e9eb5 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:10.589891 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518d90e-88c4-476b-a6a7-7e75882e9eb5 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:10.590162 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518d90e-88c4-476b-a6a7-7e75882e9eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:10.615940 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-2518d90e-88c4-476b-a6a7-7e75882e9eb5 None None] MFA Rules not processed for user `28fc34fb729f48f9aa9cf76547f3d6eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:28:10.617136 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-2518d90e-88c4-476b-a6a7-7e75882e9eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:28:10.682318 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1419/2834] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:28:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:28:10.694500 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc219056-9c03-4146-8b96-7e46bb805467 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:28:10.696301 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-cc219056-9c03-4146-8b96-7e46bb805467 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:28:10.751345 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc219056-9c03-4146-8b96-7e46bb805467 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:28:10.755694 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414f51b4-0ef1-4be3-a01f-e557c6a7d4db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:28:10.756337 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414f51b4-0ef1-4be3-a01f-e557c6a7d4db service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:10.756621 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414f51b4-0ef1-4be3-a01f-e557c6a7d4db service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:10.756894 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414f51b4-0ef1-4be3-a01f-e557c6a7d4db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:10.765044 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414f51b4-0ef1-4be3-a01f-e557c6a7d4db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:28:10.766465 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414f51b4-0ef1-4be3-a01f-e557c6a7d4db service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:28:10.775153 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1416/2835] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:28:10 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:28:20.679183 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=53,keystone:INSERT=3 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:28:20.771283 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:28:34.737685 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518d90e-88c4-476b-a6a7-7e75882e9eb5 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:28:34.799328 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518d90e-88c4-476b-a6a7-7e75882e9eb5 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:28:34.804729 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7b5425-e0f2-464c-b2c7-5c5fcdc1b92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:28:34.805598 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b5425-e0f2-464c-b2c7-5c5fcdc1b92c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:34.805883 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b5425-e0f2-464c-b2c7-5c5fcdc1b92c admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:34.806160 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b5425-e0f2-464c-b2c7-5c5fcdc1b92c admin admin] PATH_INFO: `/v3/users/28fc34fb729f48f9aa9cf76547f3d6eb` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:34.807642 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7b5425-e0f2-464c-b2c7-5c5fcdc1b92c admin admin] RBAC: Authorizing `identity:delete_user(user_id=28fc34fb729f48f9aa9cf76547f3d6eb)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:28:34.809549 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7b5425-e0f2-464c-b2c7-5c5fcdc1b92c admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:28:34.940316 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-1e7b5425-e0f2-464c-b2c7-5c5fcdc1b92c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28fc34fb729f48f9aa9cf76547f3d6eb'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:34.989250 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-1e7b5425-e0f2-464c-b2c7-5c5fcdc1b92c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28fc34fb729f48f9aa9cf76547f3d6eb'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:34.989917 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-1e7b5425-e0f2-464c-b2c7-5c5fcdc1b92c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28fc34fb729f48f9aa9cf76547f3d6eb'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:34.990537 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-1e7b5425-e0f2-464c-b2c7-5c5fcdc1b92c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28fc34fb729f48f9aa9cf76547f3d6eb'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:34.997627 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-1e7b5425-e0f2-464c-b2c7-5c5fcdc1b92c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28fc34fb729f48f9aa9cf76547f3d6eb'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:35.014832 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1420/2836] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 17:28:34 2022] DELETE /identity/v3/users/28fc34fb729f48f9aa9cf76547f3d6eb => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:28:35.019299 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86dad0c-4add-456a-94e5-88a2ad8ae429 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:35.019813 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86dad0c-4add-456a-94e5-88a2ad8ae429 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:35.020105 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86dad0c-4add-456a-94e5-88a2ad8ae429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:35.037652 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-c86dad0c-4add-456a-94e5-88a2ad8ae429 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:28:35.038846 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-c86dad0c-4add-456a-94e5-88a2ad8ae429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:28:35.095743 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1417/2837] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:28:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:28:35.105873 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7b5425-e0f2-464c-b2c7-5c5fcdc1b92c admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:28:35.107277 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-1e7b5425-e0f2-464c-b2c7-5c5fcdc1b92c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:28:35.164959 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7b5425-e0f2-464c-b2c7-5c5fcdc1b92c admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:28:35.168647 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48cf642-eb57-4068-b7c0-0dae1d78f30c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:28:35.169299 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48cf642-eb57-4068-b7c0-0dae1d78f30c service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:35.169584 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48cf642-eb57-4068-b7c0-0dae1d78f30c service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:35.169967 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48cf642-eb57-4068-b7c0-0dae1d78f30c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:35.179110 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48cf642-eb57-4068-b7c0-0dae1d78f30c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:28:35.180584 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48cf642-eb57-4068-b7c0-0dae1d78f30c service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:28:35.190888 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1421/2838] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:28:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:28:35.613462 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86dad0c-4add-456a-94e5-88a2ad8ae429 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:28:35.614951 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-c86dad0c-4add-456a-94e5-88a2ad8ae429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:28:35.627320 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86dad0c-4add-456a-94e5-88a2ad8ae429 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:28:35.632885 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5438d74c-d21f-4081-afd1-053a55254782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:28:35.633757 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5438d74c-d21f-4081-afd1-053a55254782 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:35.634045 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5438d74c-d21f-4081-afd1-053a55254782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:35.634323 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5438d74c-d21f-4081-afd1-053a55254782 admin admin] PATH_INFO: `/v3/projects/c9a2a6104d92436591f78a4d07c77592` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:35.635689 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5438d74c-d21f-4081-afd1-053a55254782 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9a2a6104d92436591f78a4d07c77592)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:28:35.637477 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5438d74c-d21f-4081-afd1-053a55254782 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:28:35.732701 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-5438d74c-d21f-4081-afd1-053a55254782 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9a2a6104d92436591f78a4d07c77592', 'request_id': 'req-5438d74c-d21f-4081-afd1-053a55254782'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:35.738173 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-5438d74c-d21f-4081-afd1-053a55254782 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9a2a6104d92436591f78a4d07c77592', 'request_id': 'req-5438d74c-d21f-4081-afd1-053a55254782'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:35.744455 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-5438d74c-d21f-4081-afd1-053a55254782 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9a2a6104d92436591f78a4d07c77592', 'request_id': 'req-5438d74c-d21f-4081-afd1-053a55254782'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:35.794876 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-5438d74c-d21f-4081-afd1-053a55254782 admin admin] The token cache is being invalidate because project c9a2a6104d92436591f78a4d07c77592 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 17:28:35.795152 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-5438d74c-d21f-4081-afd1-053a55254782 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:35.795904 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-5438d74c-d21f-4081-afd1-053a55254782 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:28:35.797469 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1418/2839] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 17:28:35 2022] DELETE /identity/v3/projects/c9a2a6104d92436591f78a4d07c77592 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:28:41.614241 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a3723e-7fa7-4552-ba48-45bcbe68eacc None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:28:41.614737 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a3723e-7fa7-4552-ba48-45bcbe68eacc None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:28:41.614924 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a3723e-7fa7-4552-ba48-45bcbe68eacc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:28:41.632825 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-e6a3723e-7fa7-4552-ba48-45bcbe68eacc None None] MFA Rules not processed for user `a708e1f37fd5453198e32ec6427afcaa`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:28:41.634013 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-e6a3723e-7fa7-4552-ba48-45bcbe68eacc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:28:41.650538 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:28:41.689678 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1422/2840] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:28:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:28:45.791268 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:28:51.715875 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=15 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:29:31.873564 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ea891-5e10-4817-92a8-c0390fff946c None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:29:31.874157 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ea891-5e10-4817-92a8-c0390fff946c None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:29:31.874157 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ea891-5e10-4817-92a8-c0390fff946c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:29:31.893411 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-a53ea891-5e10-4817-92a8-c0390fff946c None None] MFA Rules not processed for user `e1fc02859caf4ac09a85a4e5f962c14b`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:29:31.894617 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a53ea891-5e10-4817-92a8-c0390fff946c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:29:31.952204 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1419/2841] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:29:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 17:29:41.945273 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=25 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:29:55.855893 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a3723e-7fa7-4552-ba48-45bcbe68eacc None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:29:55.857376 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-e6a3723e-7fa7-4552-ba48-45bcbe68eacc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:29:55.932583 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a3723e-7fa7-4552-ba48-45bcbe68eacc None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:29:55.937894 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05bd4d9-af2b-4430-8dc3-1837c21d6aa0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70f704a231a5474e8067a6c2dd88b355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:29:55.938550 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05bd4d9-af2b-4430-8dc3-1837c21d6aa0 service placement] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:29:55.938836 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05bd4d9-af2b-4430-8dc3-1837c21d6aa0 service placement] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:29:55.939114 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05bd4d9-af2b-4430-8dc3-1837c21d6aa0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:29:55.941045 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-e05bd4d9-af2b-4430-8dc3-1837c21d6aa0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:29:55.957097 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05bd4d9-af2b-4430-8dc3-1837c21d6aa0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:29:55.957097 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05bd4d9-af2b-4430-8dc3-1837c21d6aa0 service placement] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:29:55.957268 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ea891-5e10-4817-92a8-c0390fff946c None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:29:55.958913 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-a53ea891-5e10-4817-92a8-c0390fff946c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:29:55.969225 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1423/2842] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:29:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:29:56.029367 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ea891-5e10-4817-92a8-c0390fff946c None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:29:56.033876 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd95a6c-5626-4781-a466-8370e8b45cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:29:56.034759 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd95a6c-5626-4781-a466-8370e8b45cfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:29:56.035065 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd95a6c-5626-4781-a466-8370e8b45cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:29:56.035342 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd95a6c-5626-4781-a466-8370e8b45cfa admin admin] PATH_INFO: `/v3/users/a708e1f37fd5453198e32ec6427afcaa` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:29:56.036629 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd95a6c-5626-4781-a466-8370e8b45cfa admin admin] RBAC: Authorizing `identity:delete_user(user_id=a708e1f37fd5453198e32ec6427afcaa)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:29:56.038530 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd95a6c-5626-4781-a466-8370e8b45cfa admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:29:56.277786 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-6cd95a6c-5626-4781-a466-8370e8b45cfa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a708e1f37fd5453198e32ec6427afcaa'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:29:56.354756 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-6cd95a6c-5626-4781-a466-8370e8b45cfa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a708e1f37fd5453198e32ec6427afcaa'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:29:56.362003 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-6cd95a6c-5626-4781-a466-8370e8b45cfa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a708e1f37fd5453198e32ec6427afcaa'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:29:56.373615 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-6cd95a6c-5626-4781-a466-8370e8b45cfa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a708e1f37fd5453198e32ec6427afcaa'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:29:56.374240 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-6cd95a6c-5626-4781-a466-8370e8b45cfa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a708e1f37fd5453198e32ec6427afcaa'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:29:56.380124 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1420/2843] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 17:29:55 2022] DELETE /identity/v3/users/a708e1f37fd5453198e32ec6427afcaa => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 17:29:56.630214 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05bd4d9-af2b-4430-8dc3-1837c21d6aa0 service placement] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:29:56.641551 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05bd4d9-af2b-4430-8dc3-1837c21d6aa0 service placement] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:29:56.699591 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7b217f-4cab-48ae-8529-0fc578f8b513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:29:56.700465 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b217f-4cab-48ae-8529-0fc578f8b513 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:29:56.700752 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b217f-4cab-48ae-8529-0fc578f8b513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:29:56.701032 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b217f-4cab-48ae-8529-0fc578f8b513 admin admin] PATH_INFO: `/v3/projects/60611329802847dd9da7d4d230ab32a4` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:29:56.702372 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7b217f-4cab-48ae-8529-0fc578f8b513 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60611329802847dd9da7d4d230ab32a4)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:29:56.704315 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7b217f-4cab-48ae-8529-0fc578f8b513 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:29:56.839963 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-0f7b217f-4cab-48ae-8529-0fc578f8b513 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60611329802847dd9da7d4d230ab32a4', 'request_id': 'req-0f7b217f-4cab-48ae-8529-0fc578f8b513'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:29:56.847993 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-0f7b217f-4cab-48ae-8529-0fc578f8b513 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60611329802847dd9da7d4d230ab32a4', 'request_id': 'req-0f7b217f-4cab-48ae-8529-0fc578f8b513'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:29:56.920596 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-0f7b217f-4cab-48ae-8529-0fc578f8b513 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60611329802847dd9da7d4d230ab32a4', 'request_id': 'req-0f7b217f-4cab-48ae-8529-0fc578f8b513'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:29:56.932007 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-0f7b217f-4cab-48ae-8529-0fc578f8b513 admin admin] The token cache is being invalidate because project 60611329802847dd9da7d4d230ab32a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 17:29:56.932465 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-0f7b217f-4cab-48ae-8529-0fc578f8b513 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:29:56.933358 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-0f7b217f-4cab-48ae-8529-0fc578f8b513 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:29:56.934991 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1424/2844] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 17:29:56 2022] DELETE /identity/v3/projects/60611329802847dd9da7d4d230ab32a4 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:29:56.939989 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89427449-9d5c-4a00-af64-fccf421826e1 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:29:56.944720 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89427449-9d5c-4a00-af64-fccf421826e1 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:29:56.944720 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89427449-9d5c-4a00-af64-fccf421826e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:29:56.959222 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-89427449-9d5c-4a00-af64-fccf421826e1 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:29:56.960633 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-89427449-9d5c-4a00-af64-fccf421826e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:29:57.019141 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1421/2845] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:29:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:29:57.027237 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d87bb2e-3f34-4300-86d8-b612f13a48db None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:29:57.027551 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d87bb2e-3f34-4300-86d8-b612f13a48db None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:29:57.027834 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d87bb2e-3f34-4300-86d8-b612f13a48db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:29:57.046238 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-8d87bb2e-3f34-4300-86d8-b612f13a48db None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:29:57.047451 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-8d87bb2e-3f34-4300-86d8-b612f13a48db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:29:57.055193 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1425/2846] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:29:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:29:57.060533 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89427449-9d5c-4a00-af64-fccf421826e1 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:29:57.070341 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89427449-9d5c-4a00-af64-fccf421826e1 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:29:57.075047 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26235aa-7f15-4b90-af5b-cddc260ac7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:29:57.075758 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26235aa-7f15-4b90-af5b-cddc260ac7e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:29:57.076047 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26235aa-7f15-4b90-af5b-cddc260ac7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:29:57.076327 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26235aa-7f15-4b90-af5b-cddc260ac7e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:29:57.077051 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26235aa-7f15-4b90-af5b-cddc260ac7e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 17:29:57.077357 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26235aa-7f15-4b90-af5b-cddc260ac7e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:29:57.079040 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26235aa-7f15-4b90-af5b-cddc260ac7e1 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:29:57.091017 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1422/2847] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 17:29:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 17:29:57.095378 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d87bb2e-3f34-4300-86d8-b612f13a48db None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:29:57.105823 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d87bb2e-3f34-4300-86d8-b612f13a48db None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:29:57.110544 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068850a7-e613-4b9c-a531-733e2a35ab32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:29:57.111200 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068850a7-e613-4b9c-a531-733e2a35ab32 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:29:57.111504 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068850a7-e613-4b9c-a531-733e2a35ab32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:29:57.111798 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068850a7-e613-4b9c-a531-733e2a35ab32 admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:29:57.112500 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068850a7-e613-4b9c-a531-733e2a35ab32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:29:57.114601 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068850a7-e613-4b9c-a531-733e2a35ab32 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:29:57.125428 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:29:57.253546 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1426/2848] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 17:29:57 2022] POST /identity/v3/projects => generated 365 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:29:57.257844 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26235aa-7f15-4b90-af5b-cddc260ac7e1 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:29:57.268220 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26235aa-7f15-4b90-af5b-cddc260ac7e1 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:29:57.272514 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:29:57.325520 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82275e91-e387-4f19-bc09-8fa53ba96953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:29:57.326185 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82275e91-e387-4f19-bc09-8fa53ba96953 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:29:57.326470 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82275e91-e387-4f19-bc09-8fa53ba96953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:29:57.326748 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82275e91-e387-4f19-bc09-8fa53ba96953 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:29:57.327436 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82275e91-e387-4f19-bc09-8fa53ba96953 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:29:57.329441 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82275e91-e387-4f19-bc09-8fa53ba96953 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:29:57.429297 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1423/2849] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 17:29:57 2022] POST /identity/v3/users => generated 283 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:29:57.433677 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068850a7-e613-4b9c-a531-733e2a35ab32 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:29:57.444298 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068850a7-e613-4b9c-a531-733e2a35ab32 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:29:57.449198 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e4d120-d7a5-4bd9-8745-329d74dc5b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:29:57.449860 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4d120-d7a5-4bd9-8745-329d74dc5b0f admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:29:57.450142 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4d120-d7a5-4bd9-8745-329d74dc5b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:29:57.450481 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4d120-d7a5-4bd9-8745-329d74dc5b0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:29:57.451220 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e4d120-d7a5-4bd9-8745-329d74dc5b0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:29:57.452845 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e4d120-d7a5-4bd9-8745-329d74dc5b0f admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:29:57.463738 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1427/2850] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 17:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:29:57.467534 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82275e91-e387-4f19-bc09-8fa53ba96953 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:29:57.477705 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82275e91-e387-4f19-bc09-8fa53ba96953 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:29:57.484037 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1acec1-e1e1-4796-8cd8-10f2542272f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:29:57.484693 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1acec1-e1e1-4796-8cd8-10f2542272f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:29:57.484981 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1acec1-e1e1-4796-8cd8-10f2542272f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:29:57.485260 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1acec1-e1e1-4796-8cd8-10f2542272f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:29:57.485990 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1acec1-e1e1-4796-8cd8-10f2542272f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:29:57.487644 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1acec1-e1e1-4796-8cd8-10f2542272f7 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:29:57.498502 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1424/2851] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 17:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:29:57.502692 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e4d120-d7a5-4bd9-8745-329d74dc5b0f admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:29:57.513219 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e4d120-d7a5-4bd9-8745-329d74dc5b0f admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:29:57.518337 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad91a02-5d26-450e-9325-3b1ef5b95f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:29:57.519220 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad91a02-5d26-450e-9325-3b1ef5b95f5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:29:57.519518 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad91a02-5d26-450e-9325-3b1ef5b95f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:29:57.519807 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad91a02-5d26-450e-9325-3b1ef5b95f5e admin admin] PATH_INFO: `/v3/projects/53354f718e8a47df8a0728f8e680aa6c/users/3ad2ebbe14104f1292c0c028f382118a/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:29:57.535410 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad91a02-5d26-450e-9325-3b1ef5b95f5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=53354f718e8a47df8a0728f8e680aa6c, user_id=3ad2ebbe14104f1292c0c028f382118a, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:29:57.538142 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad91a02-5d26-450e-9325-3b1ef5b95f5e admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:29:57.646386 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1428/2852] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 17:29:57 2022] PUT /identity/v3/projects/53354f718e8a47df8a0728f8e680aa6c/users/3ad2ebbe14104f1292c0c028f382118a/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:29:57.657368 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a0860-1eac-4c4f-b303-37a3aaae20c8 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:29:57.657655 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a0860-1eac-4c4f-b303-37a3aaae20c8 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:29:57.657932 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a0860-1eac-4c4f-b303-37a3aaae20c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:29:57.684993 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-629a0860-1eac-4c4f-b303-37a3aaae20c8 None None] MFA Rules not processed for user `3ad2ebbe14104f1292c0c028f382118a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:29:57.686204 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-629a0860-1eac-4c4f-b303-37a3aaae20c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:29:57.749206 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1425/2853] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:29:57 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:29:57.753646 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad91a02-5d26-450e-9325-3b1ef5b95f5e admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:29:57.814047 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad91a02-5d26-450e-9325-3b1ef5b95f5e admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:29:57.818601 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38268f74-277b-4c96-8a19-fb46303a8292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:29:57.819299 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38268f74-277b-4c96-8a19-fb46303a8292 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:29:57.819697 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38268f74-277b-4c96-8a19-fb46303a8292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:29:57.819978 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38268f74-277b-4c96-8a19-fb46303a8292 admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:29:57.820701 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38268f74-277b-4c96-8a19-fb46303a8292 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:29:57.823211 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38268f74-277b-4c96-8a19-fb46303a8292 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:29:57.916220 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1429/2854] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 17:29:57 2022] POST /identity/v3/projects => generated 361 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 17:29:57.920446 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a0860-1eac-4c4f-b303-37a3aaae20c8 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:29:57.931422 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a0860-1eac-4c4f-b303-37a3aaae20c8 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:29:57.988020 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28e3e68-023a-4223-9fb5-c636b7488646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:29:57.988695 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e3e68-023a-4223-9fb5-c636b7488646 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:29:57.988976 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e3e68-023a-4223-9fb5-c636b7488646 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:29:57.989252 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e3e68-023a-4223-9fb5-c636b7488646 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:29:57.989934 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28e3e68-023a-4223-9fb5-c636b7488646 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:29:57.991889 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28e3e68-023a-4223-9fb5-c636b7488646 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:29:58.052713 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1426/2855] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 17:29:57 2022] POST /identity/v3/users => generated 287 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:29:58.056935 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38268f74-277b-4c96-8a19-fb46303a8292 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:29:58.067729 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38268f74-277b-4c96-8a19-fb46303a8292 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:29:58.073011 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5a5848-9ab9-4594-9bf9-5781763589ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:29:58.073673 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5a5848-9ab9-4594-9bf9-5781763589ac admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:29:58.073959 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5a5848-9ab9-4594-9bf9-5781763589ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:29:58.074239 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5a5848-9ab9-4594-9bf9-5781763589ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:29:58.075038 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5a5848-9ab9-4594-9bf9-5781763589ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:29:58.076706 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5a5848-9ab9-4594-9bf9-5781763589ac admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:29:58.087379 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1430/2856] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 17:29:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:29:58.092289 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28e3e68-023a-4223-9fb5-c636b7488646 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:29:58.098839 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:29:58.102678 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28e3e68-023a-4223-9fb5-c636b7488646 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:29:58.107647 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3175318c-bd28-4e48-bf17-4c537b6a2666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:29:58.108530 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3175318c-bd28-4e48-bf17-4c537b6a2666 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:29:58.108817 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3175318c-bd28-4e48-bf17-4c537b6a2666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:29:58.109092 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3175318c-bd28-4e48-bf17-4c537b6a2666 admin admin] PATH_INFO: `/v3/projects/fb23d8acb2b9475b94f0df56869fc0f6/users/23e624d7f25a49bda7a1ba19dddf97c3/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:29:58.124689 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3175318c-bd28-4e48-bf17-4c537b6a2666 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb23d8acb2b9475b94f0df56869fc0f6, user_id=23e624d7f25a49bda7a1ba19dddf97c3, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:29:58.128281 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3175318c-bd28-4e48-bf17-4c537b6a2666 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:29:58.223268 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1427/2857] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 17:29:58 2022] PUT /identity/v3/projects/fb23d8acb2b9475b94f0df56869fc0f6/users/23e624d7f25a49bda7a1ba19dddf97c3/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:29:58.227242 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5a5848-9ab9-4594-9bf9-5781763589ac admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:29:58.237515 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5a5848-9ab9-4594-9bf9-5781763589ac admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:29:58.294610 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde43a27-09e5-4a89-a8be-6736793a8e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:29:58.295340 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde43a27-09e5-4a89-a8be-6736793a8e4d admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:29:58.295663 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde43a27-09e5-4a89-a8be-6736793a8e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:29:58.295946 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde43a27-09e5-4a89-a8be-6736793a8e4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:29:58.296671 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde43a27-09e5-4a89-a8be-6736793a8e4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:29:58.298275 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde43a27-09e5-4a89-a8be-6736793a8e4d admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:29:58.309101 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1431/2858] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 17:29:58 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:29:58.313305 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3175318c-bd28-4e48-bf17-4c537b6a2666 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:29:58.324763 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3175318c-bd28-4e48-bf17-4c537b6a2666 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:29:58.329424 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c711b64-5d5f-46e1-bd15-52980841aa8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:29:58.330303 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c711b64-5d5f-46e1-bd15-52980841aa8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:29:58.330588 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c711b64-5d5f-46e1-bd15-52980841aa8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:29:58.330865 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c711b64-5d5f-46e1-bd15-52980841aa8d admin admin] PATH_INFO: `/v3/projects/fb23d8acb2b9475b94f0df56869fc0f6/users/23e624d7f25a49bda7a1ba19dddf97c3/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:29:58.332895 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c711b64-5d5f-46e1-bd15-52980841aa8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb23d8acb2b9475b94f0df56869fc0f6, user_id=23e624d7f25a49bda7a1ba19dddf97c3, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:29:58.335538 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c711b64-5d5f-46e1-bd15-52980841aa8d admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:29:58.345947 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1428/2859] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 17:29:58 2022] PUT /identity/v3/projects/fb23d8acb2b9475b94f0df56869fc0f6/users/23e624d7f25a49bda7a1ba19dddf97c3/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:29:58.357377 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6df76-95a2-4fd2-86e5-836fe6bee5a2 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:29:58.357671 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6df76-95a2-4fd2-86e5-836fe6bee5a2 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:29:58.357953 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6df76-95a2-4fd2-86e5-836fe6bee5a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:29:58.384915 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-b0f6df76-95a2-4fd2-86e5-836fe6bee5a2 None None] MFA Rules not processed for user `23e624d7f25a49bda7a1ba19dddf97c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:29:58.386161 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-b0f6df76-95a2-4fd2-86e5-836fe6bee5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:29:58.398393 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:29:58.454095 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1432/2860] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:29:58 2022] POST /identity/v3/auth/tokens => generated 3295 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:29:58.465410 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c711b64-5d5f-46e1-bd15-52980841aa8d admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:29:58.467076 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-6c711b64-5d5f-46e1-bd15-52980841aa8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:29:58.536206 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c711b64-5d5f-46e1-bd15-52980841aa8d admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:29:58.539851 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6e81e-33bb-4827-b4d7-bbe0566cf648 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:29:58.540495 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6e81e-33bb-4827-b4d7-bbe0566cf648 service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:29:58.541303 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6e81e-33bb-4827-b4d7-bbe0566cf648 service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:29:58.541580 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6e81e-33bb-4827-b4d7-bbe0566cf648 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:29:58.550805 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e6e81e-33bb-4827-b4d7-bbe0566cf648 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:29:58.552273 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e6e81e-33bb-4827-b4d7-bbe0566cf648 service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:29:58.562556 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1429/2861] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:29:58 2022] GET /identity/v3/auth/tokens => generated 3295 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:29:59.361078 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f6df76-95a2-4fd2-86e5-836fe6bee5a2 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:29:59.424335 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f6df76-95a2-4fd2-86e5-836fe6bee5a2 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:29:59.428875 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51f0fee-0ce3-467f-815a-92938f1878fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:29:59.429747 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51f0fee-0ce3-467f-815a-92938f1878fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:29:59.430029 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51f0fee-0ce3-467f-815a-92938f1878fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:29:59.430308 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51f0fee-0ce3-467f-815a-92938f1878fe admin admin] PATH_INFO: `/v3/users/3ad2ebbe14104f1292c0c028f382118a` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:29:59.431581 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51f0fee-0ce3-467f-815a-92938f1878fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ad2ebbe14104f1292c0c028f382118a)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:29:59.433783 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51f0fee-0ce3-467f-815a-92938f1878fe admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:29:59.596950 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-c51f0fee-0ce3-467f-815a-92938f1878fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ad2ebbe14104f1292c0c028f382118a'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:29:59.674457 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-c51f0fee-0ce3-467f-815a-92938f1878fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ad2ebbe14104f1292c0c028f382118a'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:29:59.675081 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-c51f0fee-0ce3-467f-815a-92938f1878fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ad2ebbe14104f1292c0c028f382118a'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:29:59.675738 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-c51f0fee-0ce3-467f-815a-92938f1878fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ad2ebbe14104f1292c0c028f382118a'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:29:59.684384 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-c51f0fee-0ce3-467f-815a-92938f1878fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ad2ebbe14104f1292c0c028f382118a'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:29:59.702683 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1433/2862] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 17:29:59 2022] DELETE /identity/v3/users/3ad2ebbe14104f1292c0c028f382118a => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:29:59.707242 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e69090e-eace-4d06-b58f-2df3c959d600 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:29:59.707713 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e69090e-eace-4d06-b58f-2df3c959d600 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:29:59.707824 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e69090e-eace-4d06-b58f-2df3c959d600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:29:59.724487 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-6e69090e-eace-4d06-b58f-2df3c959d600 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:29:59.725672 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-6e69090e-eace-4d06-b58f-2df3c959d600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:29:59.782574 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1430/2863] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:29:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:29:59.792619 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51f0fee-0ce3-467f-815a-92938f1878fe admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:29:59.794065 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-c51f0fee-0ce3-467f-815a-92938f1878fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:29:59.829528 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:29:59.852835 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51f0fee-0ce3-467f-815a-92938f1878fe admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:29:59.856508 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e3b253-965c-433a-827c-6da2eca3b134 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:29:59.857356 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e3b253-965c-433a-827c-6da2eca3b134 service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:29:59.857851 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e3b253-965c-433a-827c-6da2eca3b134 service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:29:59.858552 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e3b253-965c-433a-827c-6da2eca3b134 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:29:59.867615 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e3b253-965c-433a-827c-6da2eca3b134 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:29:59.869622 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e3b253-965c-433a-827c-6da2eca3b134 service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:29:59.885045 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1434/2864] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:29:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 17:30:00.270675 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e69090e-eace-4d06-b58f-2df3c959d600 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:00.272212 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-6e69090e-eace-4d06-b58f-2df3c959d600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:30:00.286381 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e69090e-eace-4d06-b58f-2df3c959d600 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:00.291905 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8bb734-8c32-442a-95b4-26132d1a9e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:00.292887 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8bb734-8c32-442a-95b4-26132d1a9e2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:00.293170 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8bb734-8c32-442a-95b4-26132d1a9e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:00.293443 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8bb734-8c32-442a-95b4-26132d1a9e2b admin admin] PATH_INFO: `/v3/projects/53354f718e8a47df8a0728f8e680aa6c` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:00.294687 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8bb734-8c32-442a-95b4-26132d1a9e2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=53354f718e8a47df8a0728f8e680aa6c)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:00.296841 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8bb734-8c32-442a-95b4-26132d1a9e2b admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:00.394685 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-0c8bb734-8c32-442a-95b4-26132d1a9e2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53354f718e8a47df8a0728f8e680aa6c', 'request_id': 'req-0c8bb734-8c32-442a-95b4-26132d1a9e2b'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:00.398334 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:30:00.401253 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-0c8bb734-8c32-442a-95b4-26132d1a9e2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53354f718e8a47df8a0728f8e680aa6c', 'request_id': 'req-0c8bb734-8c32-442a-95b4-26132d1a9e2b'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:00.407593 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-0c8bb734-8c32-442a-95b4-26132d1a9e2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53354f718e8a47df8a0728f8e680aa6c', 'request_id': 'req-0c8bb734-8c32-442a-95b4-26132d1a9e2b'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:00.491575 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-0c8bb734-8c32-442a-95b4-26132d1a9e2b admin admin] The token cache is being invalidate because project 53354f718e8a47df8a0728f8e680aa6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 17:30:00.491903 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-0c8bb734-8c32-442a-95b4-26132d1a9e2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:00.492558 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-0c8bb734-8c32-442a-95b4-26132d1a9e2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:00.494408 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1431/2865] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 17:30:00 2022] DELETE /identity/v3/projects/53354f718e8a47df8a0728f8e680aa6c => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:30:00.498383 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e3b253-965c-433a-827c-6da2eca3b134 service neutron] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:00.508360 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e3b253-965c-433a-827c-6da2eca3b134 service neutron] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:00.566775 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1c0fba-8ab1-4d1d-90c4-bf989e830224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:00.567691 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c0fba-8ab1-4d1d-90c4-bf989e830224 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:00.567974 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c0fba-8ab1-4d1d-90c4-bf989e830224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:00.568280 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c0fba-8ab1-4d1d-90c4-bf989e830224 admin admin] PATH_INFO: `/v3/users/23e624d7f25a49bda7a1ba19dddf97c3` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:00.569491 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1c0fba-8ab1-4d1d-90c4-bf989e830224 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23e624d7f25a49bda7a1ba19dddf97c3)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:00.571518 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1c0fba-8ab1-4d1d-90c4-bf989e830224 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:00.745083 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-8c1c0fba-8ab1-4d1d-90c4-bf989e830224 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23e624d7f25a49bda7a1ba19dddf97c3'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:00.817406 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-8c1c0fba-8ab1-4d1d-90c4-bf989e830224 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23e624d7f25a49bda7a1ba19dddf97c3'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:00.818029 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-8c1c0fba-8ab1-4d1d-90c4-bf989e830224 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23e624d7f25a49bda7a1ba19dddf97c3'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:00.818668 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-8c1c0fba-8ab1-4d1d-90c4-bf989e830224 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23e624d7f25a49bda7a1ba19dddf97c3'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:00.826696 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-8c1c0fba-8ab1-4d1d-90c4-bf989e830224 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23e624d7f25a49bda7a1ba19dddf97c3'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:00.848698 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1435/2866] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 17:30:00 2022] DELETE /identity/v3/users/23e624d7f25a49bda7a1ba19dddf97c3 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:30:01.299102 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8bb734-8c32-442a-95b4-26132d1a9e2b admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:01.300613 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-0c8bb734-8c32-442a-95b4-26132d1a9e2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:30:01.315460 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8bb734-8c32-442a-95b4-26132d1a9e2b admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:01.373452 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15abc638-b535-4017-8e7b-b6fd5ba75f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:01.374333 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abc638-b535-4017-8e7b-b6fd5ba75f50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:01.374614 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abc638-b535-4017-8e7b-b6fd5ba75f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:01.374886 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abc638-b535-4017-8e7b-b6fd5ba75f50 admin admin] PATH_INFO: `/v3/projects/fb23d8acb2b9475b94f0df56869fc0f6` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:01.376096 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15abc638-b535-4017-8e7b-b6fd5ba75f50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb23d8acb2b9475b94f0df56869fc0f6)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:01.378085 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15abc638-b535-4017-8e7b-b6fd5ba75f50 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:01.481873 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-15abc638-b535-4017-8e7b-b6fd5ba75f50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb23d8acb2b9475b94f0df56869fc0f6', 'request_id': 'req-15abc638-b535-4017-8e7b-b6fd5ba75f50'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:01.487662 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-15abc638-b535-4017-8e7b-b6fd5ba75f50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb23d8acb2b9475b94f0df56869fc0f6', 'request_id': 'req-15abc638-b535-4017-8e7b-b6fd5ba75f50'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:01.494860 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-15abc638-b535-4017-8e7b-b6fd5ba75f50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb23d8acb2b9475b94f0df56869fc0f6', 'request_id': 'req-15abc638-b535-4017-8e7b-b6fd5ba75f50'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:01.565487 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-15abc638-b535-4017-8e7b-b6fd5ba75f50 admin admin] The token cache is being invalidate because project fb23d8acb2b9475b94f0df56869fc0f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 17:30:01.565714 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-15abc638-b535-4017-8e7b-b6fd5ba75f50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:01.566461 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-15abc638-b535-4017-8e7b-b6fd5ba75f50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:01.568035 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1432/2867] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 17:30:01 2022] DELETE /identity/v3/projects/fb23d8acb2b9475b94f0df56869fc0f6 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:30:01.573244 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e6a54-2a4c-47a6-b5a7-d874aaa6da0b None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:01.573711 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e6a54-2a4c-47a6-b5a7-d874aaa6da0b None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:01.573949 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e6a54-2a4c-47a6-b5a7-d874aaa6da0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:01.593511 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-af8e6a54-2a4c-47a6-b5a7-d874aaa6da0b None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:30:01.594709 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-af8e6a54-2a4c-47a6-b5a7-d874aaa6da0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:30:01.641093 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:30:01.653635 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1436/2868] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:30:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 17:30:01.661689 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988bf544-f707-474b-8d70-78b5516354d7 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:01.661984 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988bf544-f707-474b-8d70-78b5516354d7 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:01.662267 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988bf544-f707-474b-8d70-78b5516354d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:01.681067 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-988bf544-f707-474b-8d70-78b5516354d7 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:30:01.682526 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-988bf544-f707-474b-8d70-78b5516354d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:30:01.690976 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1433/2869] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:30:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:30:01.695416 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8e6a54-2a4c-47a6-b5a7-d874aaa6da0b None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:01.706017 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8e6a54-2a4c-47a6-b5a7-d874aaa6da0b None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:01.710656 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c70c0a-8102-4995-81df-ccd17287d9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:01.711309 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c70c0a-8102-4995-81df-ccd17287d9b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:01.711626 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c70c0a-8102-4995-81df-ccd17287d9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:01.711951 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c70c0a-8102-4995-81df-ccd17287d9b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:01.713262 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c70c0a-8102-4995-81df-ccd17287d9b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 17:30:01.713668 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c70c0a-8102-4995-81df-ccd17287d9b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:01.715249 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c70c0a-8102-4995-81df-ccd17287d9b6 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:01.726734 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1437/2870] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 17:30:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 17:30:01.730998 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988bf544-f707-474b-8d70-78b5516354d7 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:01.740327 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988bf544-f707-474b-8d70-78b5516354d7 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:01.745070 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee56838a-f2af-46ab-a836-af25f456a3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:01.745798 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee56838a-f2af-46ab-a836-af25f456a3f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:01.746088 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee56838a-f2af-46ab-a836-af25f456a3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:01.746366 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee56838a-f2af-46ab-a836-af25f456a3f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:01.747058 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee56838a-f2af-46ab-a836-af25f456a3f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:01.748997 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee56838a-f2af-46ab-a836-af25f456a3f9 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:01.830783 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1434/2871] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 17:30:01 2022] POST /identity/v3/projects => generated 383 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:30:01.834884 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c70c0a-8102-4995-81df-ccd17287d9b6 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:01.846278 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c70c0a-8102-4995-81df-ccd17287d9b6 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:01.902169 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ae6148-e0b8-48be-aa2f-12205e1c3a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:01.902871 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae6148-e0b8-48be-aa2f-12205e1c3a58 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:01.903153 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae6148-e0b8-48be-aa2f-12205e1c3a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:01.903430 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae6148-e0b8-48be-aa2f-12205e1c3a58 admin admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:01.904145 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ae6148-e0b8-48be-aa2f-12205e1c3a58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:01.906043 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ae6148-e0b8-48be-aa2f-12205e1c3a58 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:01.995707 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1438/2872] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 17:30:01 2022] POST /identity/v3/users => generated 292 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:30:01.999900 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee56838a-f2af-46ab-a836-af25f456a3f9 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:02.011471 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee56838a-f2af-46ab-a836-af25f456a3f9 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:02.016094 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d9fe0-1440-4a65-b465-9704d64e7d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:02.016761 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d9fe0-1440-4a65-b465-9704d64e7d52 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:02.017046 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d9fe0-1440-4a65-b465-9704d64e7d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:02.017323 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d9fe0-1440-4a65-b465-9704d64e7d52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:02.018117 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04d9fe0-1440-4a65-b465-9704d64e7d52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:02.019789 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04d9fe0-1440-4a65-b465-9704d64e7d52 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:02.030491 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1435/2873] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 17:30:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:30:02.034553 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ae6148-e0b8-48be-aa2f-12205e1c3a58 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:02.044552 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ae6148-e0b8-48be-aa2f-12205e1c3a58 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:02.049394 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b7b82b-bacb-4998-bfd0-95b5af422f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:02.050141 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7b82b-bacb-4998-bfd0-95b5af422f51 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:02.050422 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7b82b-bacb-4998-bfd0-95b5af422f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:02.050695 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7b82b-bacb-4998-bfd0-95b5af422f51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:02.051428 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b7b82b-bacb-4998-bfd0-95b5af422f51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:02.053090 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b7b82b-bacb-4998-bfd0-95b5af422f51 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:02.064296 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1439/2874] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 17:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:30:02.068538 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d9fe0-1440-4a65-b465-9704d64e7d52 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:02.079368 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d9fe0-1440-4a65-b465-9704d64e7d52 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:02.084048 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e36e09-1a8b-4897-921b-0025a7e983cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:02.084912 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e36e09-1a8b-4897-921b-0025a7e983cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:02.085199 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e36e09-1a8b-4897-921b-0025a7e983cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:02.085477 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e36e09-1a8b-4897-921b-0025a7e983cf admin admin] PATH_INFO: `/v3/projects/45c0e71688974e2ea5b7b59ad6d34d46/users/26bc09975f4144c8ab426337015d42e2/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:02.099884 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e36e09-1a8b-4897-921b-0025a7e983cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=45c0e71688974e2ea5b7b59ad6d34d46, user_id=26bc09975f4144c8ab426337015d42e2, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:02.102622 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e36e09-1a8b-4897-921b-0025a7e983cf admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:02.154472 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1436/2875] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 17:30:02 2022] PUT /identity/v3/projects/45c0e71688974e2ea5b7b59ad6d34d46/users/26bc09975f4144c8ab426337015d42e2/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:30:02.171355 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7012002-434d-4f97-b354-417076968d97 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:02.171670 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7012002-434d-4f97-b354-417076968d97 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:02.171945 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7012002-434d-4f97-b354-417076968d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:02.205102 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-a7012002-434d-4f97-b354-417076968d97 None None] MFA Rules not processed for user `26bc09975f4144c8ab426337015d42e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:30:02.206853 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-a7012002-434d-4f97-b354-417076968d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:30:02.273480 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1440/2876] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:30:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:30:02.278351 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e36e09-1a8b-4897-921b-0025a7e983cf admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:02.288184 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:30:02.340106 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e36e09-1a8b-4897-921b-0025a7e983cf admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:02.344698 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20215f73-3a32-4288-993f-e35ceb96cd86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:02.345354 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20215f73-3a32-4288-993f-e35ceb96cd86 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:02.345637 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20215f73-3a32-4288-993f-e35ceb96cd86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:02.345915 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20215f73-3a32-4288-993f-e35ceb96cd86 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:02.346604 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20215f73-3a32-4288-993f-e35ceb96cd86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:02.348571 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20215f73-3a32-4288-993f-e35ceb96cd86 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:02.441945 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1437/2877] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 17:30:02 2022] POST /identity/v3/projects => generated 379 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:30:02.445942 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7012002-434d-4f97-b354-417076968d97 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:02.456936 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7012002-434d-4f97-b354-417076968d97 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:02.502731 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:30:02.514536 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176d6a8a-9158-440e-aaf3-38f3a079cd7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:02.515214 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d6a8a-9158-440e-aaf3-38f3a079cd7a admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:02.515523 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d6a8a-9158-440e-aaf3-38f3a079cd7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:02.515803 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d6a8a-9158-440e-aaf3-38f3a079cd7a admin admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:02.516494 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176d6a8a-9158-440e-aaf3-38f3a079cd7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:02.518458 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176d6a8a-9158-440e-aaf3-38f3a079cd7a admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:02.617725 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1441/2878] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 17:30:02 2022] POST /identity/v3/users => generated 290 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:30:02.621777 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20215f73-3a32-4288-993f-e35ceb96cd86 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:02.632193 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20215f73-3a32-4288-993f-e35ceb96cd86 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:02.637039 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6772b6ad-6a1a-4d87-8c38-f153aae6ddf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:02.637690 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6772b6ad-6a1a-4d87-8c38-f153aae6ddf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:02.637978 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6772b6ad-6a1a-4d87-8c38-f153aae6ddf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:02.638257 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6772b6ad-6a1a-4d87-8c38-f153aae6ddf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:02.639059 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6772b6ad-6a1a-4d87-8c38-f153aae6ddf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:02.640682 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6772b6ad-6a1a-4d87-8c38-f153aae6ddf9 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:02.652047 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1438/2879] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 17:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:30:02.656195 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176d6a8a-9158-440e-aaf3-38f3a079cd7a admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:02.667046 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176d6a8a-9158-440e-aaf3-38f3a079cd7a admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:02.671911 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f2b209-623f-480f-9142-5d9c4294bba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:02.672692 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f2b209-623f-480f-9142-5d9c4294bba5 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:02.679847 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f2b209-623f-480f-9142-5d9c4294bba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:02.679847 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f2b209-623f-480f-9142-5d9c4294bba5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:02.679847 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f2b209-623f-480f-9142-5d9c4294bba5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:02.679847 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f2b209-623f-480f-9142-5d9c4294bba5 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:02.686551 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1442/2880] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 17:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:30:02.690916 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6772b6ad-6a1a-4d87-8c38-f153aae6ddf9 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:02.701163 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6772b6ad-6a1a-4d87-8c38-f153aae6ddf9 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:02.705789 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da4b19-4c54-481c-a305-dc8860d10dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:02.706659 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da4b19-4c54-481c-a305-dc8860d10dc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:02.707410 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da4b19-4c54-481c-a305-dc8860d10dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:02.707410 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da4b19-4c54-481c-a305-dc8860d10dc4 admin admin] PATH_INFO: `/v3/projects/a568432dcd33481aa17b1becaa9a377b/users/d3bf345fc53746e3a3d6ae8cf8a7b168/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:02.722729 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da4b19-4c54-481c-a305-dc8860d10dc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a568432dcd33481aa17b1becaa9a377b, user_id=d3bf345fc53746e3a3d6ae8cf8a7b168, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:02.725444 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da4b19-4c54-481c-a305-dc8860d10dc4 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:02.789396 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1439/2881] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 17:30:02 2022] PUT /identity/v3/projects/a568432dcd33481aa17b1becaa9a377b/users/d3bf345fc53746e3a3d6ae8cf8a7b168/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:30:02.802129 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bcf424-0646-4a4e-8a06-e9080de37639 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:02.802420 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bcf424-0646-4a4e-8a06-e9080de37639 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:02.802701 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bcf424-0646-4a4e-8a06-e9080de37639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:02.829965 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-e8bcf424-0646-4a4e-8a06-e9080de37639 None None] MFA Rules not processed for user `d3bf345fc53746e3a3d6ae8cf8a7b168`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:30:02.831269 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-e8bcf424-0646-4a4e-8a06-e9080de37639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:30:02.894740 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1443/2882] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:30:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 17:30:02.899318 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da4b19-4c54-481c-a305-dc8860d10dc4 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:02.962022 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da4b19-4c54-481c-a305-dc8860d10dc4 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:02.966880 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a300deb5-8801-4815-82e5-f39413afcaa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:02.967559 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a300deb5-8801-4815-82e5-f39413afcaa6 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:02.967848 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a300deb5-8801-4815-82e5-f39413afcaa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:02.968129 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a300deb5-8801-4815-82e5-f39413afcaa6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:02.968823 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a300deb5-8801-4815-82e5-f39413afcaa6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:02.970715 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a300deb5-8801-4815-82e5-f39413afcaa6 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:03.053802 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1440/2883] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 17:30:02 2022] POST /identity/v3/projects => generated 385 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:30:03.059105 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bcf424-0646-4a4e-8a06-e9080de37639 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:03.069964 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bcf424-0646-4a4e-8a06-e9080de37639 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:03.126626 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85514b91-6b71-4ca6-839e-56a92039812f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:03.127297 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85514b91-6b71-4ca6-839e-56a92039812f admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:03.127621 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85514b91-6b71-4ca6-839e-56a92039812f admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:03.127900 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85514b91-6b71-4ca6-839e-56a92039812f admin admin] PATH_INFO: `/v3/users` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:03.128577 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85514b91-6b71-4ca6-839e-56a92039812f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:03.130581 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85514b91-6b71-4ca6-839e-56a92039812f admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:03.223526 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1444/2884] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 17:30:03 2022] POST /identity/v3/users => generated 299 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:30:03.227601 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a300deb5-8801-4815-82e5-f39413afcaa6 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:03.238456 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a300deb5-8801-4815-82e5-f39413afcaa6 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:03.243037 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531d119e-ab72-462f-8cf8-8c03c451db65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:03.244338 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d119e-ab72-462f-8cf8-8c03c451db65 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:03.244625 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d119e-ab72-462f-8cf8-8c03c451db65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:03.244985 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d119e-ab72-462f-8cf8-8c03c451db65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:03.245710 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531d119e-ab72-462f-8cf8-8c03c451db65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:03.247355 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531d119e-ab72-462f-8cf8-8c03c451db65 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:03.259059 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1441/2885] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 17:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 17:30:03.263128 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85514b91-6b71-4ca6-839e-56a92039812f admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:03.273054 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85514b91-6b71-4ca6-839e-56a92039812f admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:03.278148 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10fb708-82ba-49c0-a91d-94f9cc5732aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:03.279090 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10fb708-82ba-49c0-a91d-94f9cc5732aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:03.279407 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10fb708-82ba-49c0-a91d-94f9cc5732aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:03.279709 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10fb708-82ba-49c0-a91d-94f9cc5732aa admin admin] PATH_INFO: `/v3/projects/441e60fe84934ea3b964d86362d984fe/users/5b43e80dae704445b93b83c1403e5d97/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:03.295388 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10fb708-82ba-49c0-a91d-94f9cc5732aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=441e60fe84934ea3b964d86362d984fe, user_id=5b43e80dae704445b93b83c1403e5d97, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:03.298335 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10fb708-82ba-49c0-a91d-94f9cc5732aa admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:03.349268 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1445/2886] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 17:30:03 2022] PUT /identity/v3/projects/441e60fe84934ea3b964d86362d984fe/users/5b43e80dae704445b93b83c1403e5d97/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:30:03.353902 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531d119e-ab72-462f-8cf8-8c03c451db65 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:03.364108 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531d119e-ab72-462f-8cf8-8c03c451db65 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:03.393791 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:30:03.421511 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b58e6c4-fafa-4977-9e9c-b7ac7e773c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:03.422200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b58e6c4-fafa-4977-9e9c-b7ac7e773c70 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:03.422488 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b58e6c4-fafa-4977-9e9c-b7ac7e773c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:03.422762 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b58e6c4-fafa-4977-9e9c-b7ac7e773c70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:03.423479 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b58e6c4-fafa-4977-9e9c-b7ac7e773c70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:03.425101 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b58e6c4-fafa-4977-9e9c-b7ac7e773c70 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:03.436188 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1442/2887] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 17:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:30:03.440397 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10fb708-82ba-49c0-a91d-94f9cc5732aa admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:03.450861 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10fb708-82ba-49c0-a91d-94f9cc5732aa admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:03.455444 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74196ddf-1cac-4a1f-903b-1590ee4c649d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:03.456347 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74196ddf-1cac-4a1f-903b-1590ee4c649d admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:03.456633 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74196ddf-1cac-4a1f-903b-1590ee4c649d admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:03.456907 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74196ddf-1cac-4a1f-903b-1590ee4c649d admin admin] PATH_INFO: `/v3/projects/441e60fe84934ea3b964d86362d984fe/users/5b43e80dae704445b93b83c1403e5d97/roles/98eaadb3669f4b608d572c001742f559` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:03.458862 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74196ddf-1cac-4a1f-903b-1590ee4c649d admin admin] RBAC: Authorizing `identity:create_grant(project_id=441e60fe84934ea3b964d86362d984fe, user_id=5b43e80dae704445b93b83c1403e5d97, role_id=98eaadb3669f4b608d572c001742f559)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:03.461593 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74196ddf-1cac-4a1f-903b-1590ee4c649d admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:03.472350 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1446/2888] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 17:30:03 2022] PUT /identity/v3/projects/441e60fe84934ea3b964d86362d984fe/users/5b43e80dae704445b93b83c1403e5d97/roles/98eaadb3669f4b608d572c001742f559 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:30:03.483064 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c76e5-bdb8-418c-abf4-2f8b36c46103 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:03.483305 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c76e5-bdb8-418c-abf4-2f8b36c46103 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:03.483638 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c76e5-bdb8-418c-abf4-2f8b36c46103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:03.510087 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-2d9c76e5-bdb8-418c-abf4-2f8b36c46103 None None] MFA Rules not processed for user `5b43e80dae704445b93b83c1403e5d97`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:30:03.511294 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-2d9c76e5-bdb8-418c-abf4-2f8b36c46103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:30:03.578076 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1443/2889] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:30:03 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:30:03.589704 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74196ddf-1cac-4a1f-903b-1590ee4c649d admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:03.591444 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-74196ddf-1cac-4a1f-903b-1590ee4c649d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:30:03.602089 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:30:03.649006 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74196ddf-1cac-4a1f-903b-1590ee4c649d admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:03.653347 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8672da24-a82f-49ec-96f7-2991b6921f9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:03.653990 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8672da24-a82f-49ec-96f7-2991b6921f9c service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:03.654277 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8672da24-a82f-49ec-96f7-2991b6921f9c service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:03.654552 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8672da24-a82f-49ec-96f7-2991b6921f9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:03.711086 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8672da24-a82f-49ec-96f7-2991b6921f9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:03.712703 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8672da24-a82f-49ec-96f7-2991b6921f9c service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:03.722385 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1447/2890] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:30:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:30:05.577161 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9c76e5-bdb8-418c-abf4-2f8b36c46103 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:05.578629 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-2d9c76e5-bdb8-418c-abf4-2f8b36c46103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:30:05.591228 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9c76e5-bdb8-418c-abf4-2f8b36c46103 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:05.594892 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ebfa0d-a817-4427-9b09-2790531a8f78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:05.595560 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ebfa0d-a817-4427-9b09-2790531a8f78 service cinder] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:05.595850 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ebfa0d-a817-4427-9b09-2790531a8f78 service cinder] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:05.596129 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ebfa0d-a817-4427-9b09-2790531a8f78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:05.652077 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ebfa0d-a817-4427-9b09-2790531a8f78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:05.653503 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ebfa0d-a817-4427-9b09-2790531a8f78 service cinder] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:05.662204 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1444/2891] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:30:05 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:30:06.956777 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8672da24-a82f-49ec-96f7-2991b6921f9c service cinder] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:06.966518 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8672da24-a82f-49ec-96f7-2991b6921f9c service cinder] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:06.970289 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4376be2e-fb45-4b12-82c2-b0bf145f9cdd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:06.970934 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376be2e-fb45-4b12-82c2-b0bf145f9cdd service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:06.971236 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376be2e-fb45-4b12-82c2-b0bf145f9cdd service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:06.971515 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376be2e-fb45-4b12-82c2-b0bf145f9cdd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:06.980926 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4376be2e-fb45-4b12-82c2-b0bf145f9cdd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:06.982635 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4376be2e-fb45-4b12-82c2-b0bf145f9cdd service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:06.992126 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1448/2892] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:30:06 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:30:11.475620 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ebfa0d-a817-4427-9b09-2790531a8f78 service cinder] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:11.527189 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=100 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:30:11.537889 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ebfa0d-a817-4427-9b09-2790531a8f78 service cinder] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:11.542484 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b38013-34cd-46ca-99f0-78a45cf99236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:11.543353 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b38013-34cd-46ca-99f0-78a45cf99236 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:11.543666 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b38013-34cd-46ca-99f0-78a45cf99236 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:11.543945 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b38013-34cd-46ca-99f0-78a45cf99236 admin admin] PATH_INFO: `/v3/users/26bc09975f4144c8ab426337015d42e2` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:11.545124 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b38013-34cd-46ca-99f0-78a45cf99236 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26bc09975f4144c8ab426337015d42e2)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:11.547027 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b38013-34cd-46ca-99f0-78a45cf99236 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:11.705945 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-e6b38013-34cd-46ca-99f0-78a45cf99236 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26bc09975f4144c8ab426337015d42e2'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:11.760874 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-e6b38013-34cd-46ca-99f0-78a45cf99236 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26bc09975f4144c8ab426337015d42e2'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:11.768047 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-e6b38013-34cd-46ca-99f0-78a45cf99236 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26bc09975f4144c8ab426337015d42e2'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:11.780205 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-e6b38013-34cd-46ca-99f0-78a45cf99236 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26bc09975f4144c8ab426337015d42e2'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:11.780858 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-e6b38013-34cd-46ca-99f0-78a45cf99236 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26bc09975f4144c8ab426337015d42e2'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:11.786877 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1445/2893] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 17:30:11 2022] DELETE /identity/v3/users/26bc09975f4144c8ab426337015d42e2 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:30:11.791372 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a307722-9300-4992-9983-0e049a9b4e51 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:11.791845 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a307722-9300-4992-9983-0e049a9b4e51 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:11.791973 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a307722-9300-4992-9983-0e049a9b4e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:11.810104 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-7a307722-9300-4992-9983-0e049a9b4e51 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:30:11.811301 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-7a307722-9300-4992-9983-0e049a9b4e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:30:11.869667 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1449/2894] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:30:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:30:11.879237 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b38013-34cd-46ca-99f0-78a45cf99236 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:11.880678 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-e6b38013-34cd-46ca-99f0-78a45cf99236 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:30:11.945792 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b38013-34cd-46ca-99f0-78a45cf99236 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:11.950192 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae2259-2b46-48dd-b5e1-a9dc90625761 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:11.950875 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae2259-2b46-48dd-b5e1-a9dc90625761 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:11.951158 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae2259-2b46-48dd-b5e1-a9dc90625761 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:11.956354 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae2259-2b46-48dd-b5e1-a9dc90625761 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:11.960565 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fae2259-2b46-48dd-b5e1-a9dc90625761 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:11.962145 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fae2259-2b46-48dd-b5e1-a9dc90625761 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:11.971972 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1446/2895] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:30:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:30:12.413618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a307722-9300-4992-9983-0e049a9b4e51 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:12.415031 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-7a307722-9300-4992-9983-0e049a9b4e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:30:12.430077 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a307722-9300-4992-9983-0e049a9b4e51 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:12.435442 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab59de92-6ecf-4e56-9b09-82ec9d2a13b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:12.437192 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59de92-6ecf-4e56-9b09-82ec9d2a13b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:12.437567 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59de92-6ecf-4e56-9b09-82ec9d2a13b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:12.437744 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59de92-6ecf-4e56-9b09-82ec9d2a13b2 admin admin] PATH_INFO: `/v3/projects/45c0e71688974e2ea5b7b59ad6d34d46` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:12.438944 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab59de92-6ecf-4e56-9b09-82ec9d2a13b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45c0e71688974e2ea5b7b59ad6d34d46)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:12.441113 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab59de92-6ecf-4e56-9b09-82ec9d2a13b2 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:12.564684 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:30:12.567897 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-ab59de92-6ecf-4e56-9b09-82ec9d2a13b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45c0e71688974e2ea5b7b59ad6d34d46', 'request_id': 'req-ab59de92-6ecf-4e56-9b09-82ec9d2a13b2'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:12.574239 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-ab59de92-6ecf-4e56-9b09-82ec9d2a13b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45c0e71688974e2ea5b7b59ad6d34d46', 'request_id': 'req-ab59de92-6ecf-4e56-9b09-82ec9d2a13b2'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:12.640512 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-ab59de92-6ecf-4e56-9b09-82ec9d2a13b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45c0e71688974e2ea5b7b59ad6d34d46', 'request_id': 'req-ab59de92-6ecf-4e56-9b09-82ec9d2a13b2'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:12.652109 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-ab59de92-6ecf-4e56-9b09-82ec9d2a13b2 admin admin] The token cache is being invalidate because project 45c0e71688974e2ea5b7b59ad6d34d46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 17:30:12.652600 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-ab59de92-6ecf-4e56-9b09-82ec9d2a13b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:12.653440 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-ab59de92-6ecf-4e56-9b09-82ec9d2a13b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:12.655070 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1450/2896] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 17:30:12 2022] DELETE /identity/v3/projects/45c0e71688974e2ea5b7b59ad6d34d46 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:30:12.659200 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae2259-2b46-48dd-b5e1-a9dc90625761 service neutron] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:12.670569 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae2259-2b46-48dd-b5e1-a9dc90625761 service neutron] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:12.727356 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b577036-dca7-4100-b57d-2027f0b2bad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:12.728492 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b577036-dca7-4100-b57d-2027f0b2bad8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:12.728777 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b577036-dca7-4100-b57d-2027f0b2bad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:12.729056 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b577036-dca7-4100-b57d-2027f0b2bad8 admin admin] PATH_INFO: `/v3/users/d3bf345fc53746e3a3d6ae8cf8a7b168` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:12.730254 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b577036-dca7-4100-b57d-2027f0b2bad8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3bf345fc53746e3a3d6ae8cf8a7b168)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:12.732215 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b577036-dca7-4100-b57d-2027f0b2bad8 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:12.748070 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:30:12.928608 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-4b577036-dca7-4100-b57d-2027f0b2bad8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3bf345fc53746e3a3d6ae8cf8a7b168'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:12.999801 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-4b577036-dca7-4100-b57d-2027f0b2bad8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3bf345fc53746e3a3d6ae8cf8a7b168'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:13.007190 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-4b577036-dca7-4100-b57d-2027f0b2bad8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3bf345fc53746e3a3d6ae8cf8a7b168'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:13.019582 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-4b577036-dca7-4100-b57d-2027f0b2bad8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3bf345fc53746e3a3d6ae8cf8a7b168'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:13.020227 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-4b577036-dca7-4100-b57d-2027f0b2bad8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3bf345fc53746e3a3d6ae8cf8a7b168'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:13.026662 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1447/2897] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 17:30:12 2022] DELETE /identity/v3/users/d3bf345fc53746e3a3d6ae8cf8a7b168 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:30:13.430733 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab59de92-6ecf-4e56-9b09-82ec9d2a13b2 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:13.432824 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-ab59de92-6ecf-4e56-9b09-82ec9d2a13b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:30:13.446051 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab59de92-6ecf-4e56-9b09-82ec9d2a13b2 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:13.505655 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281488de-bf23-4144-aa81-383779957897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:13.506511 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281488de-bf23-4144-aa81-383779957897 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:13.506790 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281488de-bf23-4144-aa81-383779957897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:13.507108 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281488de-bf23-4144-aa81-383779957897 admin admin] PATH_INFO: `/v3/projects/a568432dcd33481aa17b1becaa9a377b` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:13.508448 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281488de-bf23-4144-aa81-383779957897 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a568432dcd33481aa17b1becaa9a377b)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:13.510572 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281488de-bf23-4144-aa81-383779957897 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:13.612681 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-281488de-bf23-4144-aa81-383779957897 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a568432dcd33481aa17b1becaa9a377b', 'request_id': 'req-281488de-bf23-4144-aa81-383779957897'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:13.619369 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-281488de-bf23-4144-aa81-383779957897 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a568432dcd33481aa17b1becaa9a377b', 'request_id': 'req-281488de-bf23-4144-aa81-383779957897'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:13.715199 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-281488de-bf23-4144-aa81-383779957897 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a568432dcd33481aa17b1becaa9a377b', 'request_id': 'req-281488de-bf23-4144-aa81-383779957897'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:13.726501 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-281488de-bf23-4144-aa81-383779957897 admin admin] The token cache is being invalidate because project a568432dcd33481aa17b1becaa9a377b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 17:30:13.726788 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-281488de-bf23-4144-aa81-383779957897 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:13.727427 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-281488de-bf23-4144-aa81-383779957897 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:13.729021 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1451/2898] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 17:30:13 2022] DELETE /identity/v3/projects/a568432dcd33481aa17b1becaa9a377b => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:30:13.734026 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b577036-dca7-4100-b57d-2027f0b2bad8 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:13.735762 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-4b577036-dca7-4100-b57d-2027f0b2bad8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:30:13.749532 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b577036-dca7-4100-b57d-2027f0b2bad8 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:13.805015 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef436ec-e8fd-4e8b-a447-e5dad8aa2d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:13.805936 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef436ec-e8fd-4e8b-a447-e5dad8aa2d2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:13.806247 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef436ec-e8fd-4e8b-a447-e5dad8aa2d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:13.806521 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef436ec-e8fd-4e8b-a447-e5dad8aa2d2f admin admin] PATH_INFO: `/v3/users/5b43e80dae704445b93b83c1403e5d97` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:13.808315 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef436ec-e8fd-4e8b-a447-e5dad8aa2d2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b43e80dae704445b93b83c1403e5d97)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:13.810487 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef436ec-e8fd-4e8b-a447-e5dad8aa2d2f admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:13.986870 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-4ef436ec-e8fd-4e8b-a447-e5dad8aa2d2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b43e80dae704445b93b83c1403e5d97'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:14.069089 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-4ef436ec-e8fd-4e8b-a447-e5dad8aa2d2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b43e80dae704445b93b83c1403e5d97'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:14.076753 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-4ef436ec-e8fd-4e8b-a447-e5dad8aa2d2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b43e80dae704445b93b83c1403e5d97'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:14.089660 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-4ef436ec-e8fd-4e8b-a447-e5dad8aa2d2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b43e80dae704445b93b83c1403e5d97'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:14.090241 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-4ef436ec-e8fd-4e8b-a447-e5dad8aa2d2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b43e80dae704445b93b83c1403e5d97'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:14.096141 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1448/2899] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 17:30:13 2022] DELETE /identity/v3/users/5b43e80dae704445b93b83c1403e5d97 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:30:14.516392 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281488de-bf23-4144-aa81-383779957897 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:14.517854 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-281488de-bf23-4144-aa81-383779957897 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:30:14.531094 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281488de-bf23-4144-aa81-383779957897 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:14.589792 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5c299e-dccf-402f-90e5-d6d48e82b6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:14.590661 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5c299e-dccf-402f-90e5-d6d48e82b6cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:14.590949 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5c299e-dccf-402f-90e5-d6d48e82b6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:14.591228 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5c299e-dccf-402f-90e5-d6d48e82b6cc admin admin] PATH_INFO: `/v3/projects/441e60fe84934ea3b964d86362d984fe` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:14.592456 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5c299e-dccf-402f-90e5-d6d48e82b6cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=441e60fe84934ea3b964d86362d984fe)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:14.594363 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5c299e-dccf-402f-90e5-d6d48e82b6cc admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:14.600756 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:30:14.688026 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-9f5c299e-dccf-402f-90e5-d6d48e82b6cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '441e60fe84934ea3b964d86362d984fe', 'request_id': 'req-9f5c299e-dccf-402f-90e5-d6d48e82b6cc'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:14.694176 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-9f5c299e-dccf-402f-90e5-d6d48e82b6cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '441e60fe84934ea3b964d86362d984fe', 'request_id': 'req-9f5c299e-dccf-402f-90e5-d6d48e82b6cc'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:14.749146 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-9f5c299e-dccf-402f-90e5-d6d48e82b6cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '441e60fe84934ea3b964d86362d984fe', 'request_id': 'req-9f5c299e-dccf-402f-90e5-d6d48e82b6cc'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:14.759973 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-9f5c299e-dccf-402f-90e5-d6d48e82b6cc admin admin] The token cache is being invalidate because project 441e60fe84934ea3b964d86362d984fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 17:30:14.760283 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-9f5c299e-dccf-402f-90e5-d6d48e82b6cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:14.761605 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-9f5c299e-dccf-402f-90e5-d6d48e82b6cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:14.763172 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1452/2900] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 17:30:14 2022] DELETE /identity/v3/projects/441e60fe84934ea3b964d86362d984fe => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:30:14.768602 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d4100-0079-4cb0-b4e7-7284ef81b8c1 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:14.769048 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d4100-0079-4cb0-b4e7-7284ef81b8c1 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:14.769408 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d4100-0079-4cb0-b4e7-7284ef81b8c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:14.789104 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-8a0d4100-0079-4cb0-b4e7-7284ef81b8c1 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:30:14.790310 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-8a0d4100-0079-4cb0-b4e7-7284ef81b8c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:30:14.798837 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:30:14.849495 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1449/2901] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:30:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:30:14.857341 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb2f97-76a4-42db-a89e-ab3363f2f581 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:14.857627 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb2f97-76a4-42db-a89e-ab3363f2f581 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:14.857910 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb2f97-76a4-42db-a89e-ab3363f2f581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:14.876423 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-a5fb2f97-76a4-42db-a89e-ab3363f2f581 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:30:14.877623 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-a5fb2f97-76a4-42db-a89e-ab3363f2f581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:30:14.886273 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1453/2902] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:30:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:30:14.890462 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0d4100-0079-4cb0-b4e7-7284ef81b8c1 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:14.900582 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0d4100-0079-4cb0-b4e7-7284ef81b8c1 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:14.906342 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a42b116-e8f8-4c94-b71e-bf6a447899c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:14.906993 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42b116-e8f8-4c94-b71e-bf6a447899c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:14.907280 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42b116-e8f8-4c94-b71e-bf6a447899c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:14.907594 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42b116-e8f8-4c94-b71e-bf6a447899c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:14.908319 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a42b116-e8f8-4c94-b71e-bf6a447899c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 17:30:14.908625 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a42b116-e8f8-4c94-b71e-bf6a447899c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:14.910204 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a42b116-e8f8-4c94-b71e-bf6a447899c3 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:14.922199 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1450/2903] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 17:30:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 17:30:14.926489 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb2f97-76a4-42db-a89e-ab3363f2f581 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:14.936648 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb2f97-76a4-42db-a89e-ab3363f2f581 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:14.941346 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a761a-6c90-4e9d-9dcd-fa40e0ff56d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:14.942000 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a761a-6c90-4e9d-9dcd-fa40e0ff56d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:14.942284 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a761a-6c90-4e9d-9dcd-fa40e0ff56d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:14.942560 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a761a-6c90-4e9d-9dcd-fa40e0ff56d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:14.943253 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8a761a-6c90-4e9d-9dcd-fa40e0ff56d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:14.945184 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8a761a-6c90-4e9d-9dcd-fa40e0ff56d2 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:15.054320 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1454/2904] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 17:30:14 2022] POST /identity/v3/projects => generated 373 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:30:15.059134 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a42b116-e8f8-4c94-b71e-bf6a447899c3 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:15.069117 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a42b116-e8f8-4c94-b71e-bf6a447899c3 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:15.125077 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1a3a2b-9984-41aa-94cb-111d971a8e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:15.125763 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a3a2b-9984-41aa-94cb-111d971a8e66 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:15.126046 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a3a2b-9984-41aa-94cb-111d971a8e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:15.126323 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a3a2b-9984-41aa-94cb-111d971a8e66 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:15.127019 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1a3a2b-9984-41aa-94cb-111d971a8e66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:15.128983 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1a3a2b-9984-41aa-94cb-111d971a8e66 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:15.258779 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1451/2905] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 17:30:15 2022] POST /identity/v3/users => generated 287 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:30:15.262967 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a761a-6c90-4e9d-9dcd-fa40e0ff56d2 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:15.273561 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a761a-6c90-4e9d-9dcd-fa40e0ff56d2 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:15.278199 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8607db46-b722-454d-8ef2-28b3ea57cfb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:15.278867 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8607db46-b722-454d-8ef2-28b3ea57cfb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:15.279179 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8607db46-b722-454d-8ef2-28b3ea57cfb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:15.279431 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8607db46-b722-454d-8ef2-28b3ea57cfb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:15.280199 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8607db46-b722-454d-8ef2-28b3ea57cfb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:15.281837 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8607db46-b722-454d-8ef2-28b3ea57cfb1 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:15.292860 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1455/2906] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 17:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 17:30:15.296882 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1a3a2b-9984-41aa-94cb-111d971a8e66 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:15.306989 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1a3a2b-9984-41aa-94cb-111d971a8e66 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:15.312531 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453daac6-d208-42bb-91e3-5732804190a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:15.313173 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453daac6-d208-42bb-91e3-5732804190a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:15.313458 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453daac6-d208-42bb-91e3-5732804190a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:15.313751 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453daac6-d208-42bb-91e3-5732804190a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:15.314461 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453daac6-d208-42bb-91e3-5732804190a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:15.316112 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453daac6-d208-42bb-91e3-5732804190a2 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:15.326172 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1452/2907] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 17:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:30:15.330209 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8607db46-b722-454d-8ef2-28b3ea57cfb1 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:15.341159 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8607db46-b722-454d-8ef2-28b3ea57cfb1 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:15.345668 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb17ec7-9242-4541-b6b5-132061a31c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:15.346545 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb17ec7-9242-4541-b6b5-132061a31c84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:15.346831 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb17ec7-9242-4541-b6b5-132061a31c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:15.347109 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb17ec7-9242-4541-b6b5-132061a31c84 admin admin] PATH_INFO: `/v3/projects/21eeb50f71e644948b2321555d1f1e2c/users/aabbd7821ea84f5b89fdc01d7519c07b/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:15.362076 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb17ec7-9242-4541-b6b5-132061a31c84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21eeb50f71e644948b2321555d1f1e2c, user_id=aabbd7821ea84f5b89fdc01d7519c07b, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:15.364776 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb17ec7-9242-4541-b6b5-132061a31c84 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:15.458053 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1456/2908] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 17:30:15 2022] PUT /identity/v3/projects/21eeb50f71e644948b2321555d1f1e2c/users/aabbd7821ea84f5b89fdc01d7519c07b/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:30:15.470115 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26319e83-ebe0-401f-b2e2-60dcf2cdec29 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:15.470408 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26319e83-ebe0-401f-b2e2-60dcf2cdec29 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:15.470718 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26319e83-ebe0-401f-b2e2-60dcf2cdec29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:15.498193 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-26319e83-ebe0-401f-b2e2-60dcf2cdec29 None None] MFA Rules not processed for user `aabbd7821ea84f5b89fdc01d7519c07b`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:30:15.499408 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-26319e83-ebe0-401f-b2e2-60dcf2cdec29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:30:15.567981 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1453/2909] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:30:15 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:30:15.579170 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb17ec7-9242-4541-b6b5-132061a31c84 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:15.580865 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-cfb17ec7-9242-4541-b6b5-132061a31c84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:30:15.638103 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb17ec7-9242-4541-b6b5-132061a31c84 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:15.641748 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b1ede-00ba-4e37-943d-59107bdfe230 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:15.642387 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b1ede-00ba-4e37-943d-59107bdfe230 service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:15.642671 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b1ede-00ba-4e37-943d-59107bdfe230 service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:15.642947 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b1ede-00ba-4e37-943d-59107bdfe230 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:15.651666 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10b1ede-00ba-4e37-943d-59107bdfe230 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:15.653130 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10b1ede-00ba-4e37-943d-59107bdfe230 service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:15.662238 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1457/2910] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:30:15 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:30:15.685794 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26319e83-ebe0-401f-b2e2-60dcf2cdec29 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:15.695278 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:30:15.746769 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26319e83-ebe0-401f-b2e2-60dcf2cdec29 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:15.751427 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d6a2f-7c68-4635-b25c-54db52df9547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:15.752311 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d6a2f-7c68-4635-b25c-54db52df9547 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:15.752593 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d6a2f-7c68-4635-b25c-54db52df9547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:15.752868 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d6a2f-7c68-4635-b25c-54db52df9547 admin admin] PATH_INFO: `/v3/users/aabbd7821ea84f5b89fdc01d7519c07b` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:15.754176 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2d6a2f-7c68-4635-b25c-54db52df9547 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aabbd7821ea84f5b89fdc01d7519c07b)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:15.756424 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2d6a2f-7c68-4635-b25c-54db52df9547 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:15.905851 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-5a2d6a2f-7c68-4635-b25c-54db52df9547 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aabbd7821ea84f5b89fdc01d7519c07b'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:15.954356 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-5a2d6a2f-7c68-4635-b25c-54db52df9547 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aabbd7821ea84f5b89fdc01d7519c07b'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:15.961366 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-5a2d6a2f-7c68-4635-b25c-54db52df9547 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aabbd7821ea84f5b89fdc01d7519c07b'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:15.974693 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-5a2d6a2f-7c68-4635-b25c-54db52df9547 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aabbd7821ea84f5b89fdc01d7519c07b'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:15.975450 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-5a2d6a2f-7c68-4635-b25c-54db52df9547 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aabbd7821ea84f5b89fdc01d7519c07b'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:15.982115 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1454/2911] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 17:30:15 2022] DELETE /identity/v3/users/aabbd7821ea84f5b89fdc01d7519c07b => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:30:15.986611 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed8932-334c-499b-8b28-25598a04a028 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:15.990652 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed8932-334c-499b-8b28-25598a04a028 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:15.990652 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed8932-334c-499b-8b28-25598a04a028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:16.006406 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-59ed8932-334c-499b-8b28-25598a04a028 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:30:16.007654 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-59ed8932-334c-499b-8b28-25598a04a028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:30:16.036429 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:30:16.068044 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1458/2912] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:30:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:30:16.077833 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d6a2f-7c68-4635-b25c-54db52df9547 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:16.079260 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-5a2d6a2f-7c68-4635-b25c-54db52df9547 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:30:16.138129 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d6a2f-7c68-4635-b25c-54db52df9547 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:16.141804 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b1aeae-a525-407b-aa1a-10d044a91146 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:16.142459 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1aeae-a525-407b-aa1a-10d044a91146 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:16.142745 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1aeae-a525-407b-aa1a-10d044a91146 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:16.143022 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1aeae-a525-407b-aa1a-10d044a91146 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:16.153197 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b1aeae-a525-407b-aa1a-10d044a91146 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:16.154683 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b1aeae-a525-407b-aa1a-10d044a91146 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:16.164486 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1455/2913] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:30:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:30:16.539131 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ed8932-334c-499b-8b28-25598a04a028 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:16.540569 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-59ed8932-334c-499b-8b28-25598a04a028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:30:16.554420 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ed8932-334c-499b-8b28-25598a04a028 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:16.560670 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae41b849-be25-42a5-a929-fca31faeca49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:16.561539 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae41b849-be25-42a5-a929-fca31faeca49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:16.561832 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae41b849-be25-42a5-a929-fca31faeca49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:16.562115 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae41b849-be25-42a5-a929-fca31faeca49 admin admin] PATH_INFO: `/v3/projects/21eeb50f71e644948b2321555d1f1e2c` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:16.563311 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae41b849-be25-42a5-a929-fca31faeca49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21eeb50f71e644948b2321555d1f1e2c)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:16.565268 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae41b849-be25-42a5-a929-fca31faeca49 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:16.674893 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-ae41b849-be25-42a5-a929-fca31faeca49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21eeb50f71e644948b2321555d1f1e2c', 'request_id': 'req-ae41b849-be25-42a5-a929-fca31faeca49'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:16.681106 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-ae41b849-be25-42a5-a929-fca31faeca49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21eeb50f71e644948b2321555d1f1e2c', 'request_id': 'req-ae41b849-be25-42a5-a929-fca31faeca49'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:16.726378 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-ae41b849-be25-42a5-a929-fca31faeca49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21eeb50f71e644948b2321555d1f1e2c', 'request_id': 'req-ae41b849-be25-42a5-a929-fca31faeca49'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:16.737084 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-ae41b849-be25-42a5-a929-fca31faeca49 admin admin] The token cache is being invalidate because project 21eeb50f71e644948b2321555d1f1e2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 17:30:16.737373 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-ae41b849-be25-42a5-a929-fca31faeca49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:16.738000 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-ae41b849-be25-42a5-a929-fca31faeca49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:16.739560 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1459/2914] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 17:30:16 2022] DELETE /identity/v3/projects/21eeb50f71e644948b2321555d1f1e2c => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:30:16.744432 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c1d79-0ccd-40ca-91c7-db0d3293eba1 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:16.744726 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c1d79-0ccd-40ca-91c7-db0d3293eba1 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:16.745179 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c1d79-0ccd-40ca-91c7-db0d3293eba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:16.763317 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-010c1d79-0ccd-40ca-91c7-db0d3293eba1 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:30:16.764575 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-010c1d79-0ccd-40ca-91c7-db0d3293eba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:30:16.790942 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:30:16.825567 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1456/2915] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:30:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:30:16.833353 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1664730-b7bc-4f19-ba17-1dfbfe3e610e None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:16.833658 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1664730-b7bc-4f19-ba17-1dfbfe3e610e None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:16.833924 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1664730-b7bc-4f19-ba17-1dfbfe3e610e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:16.855749 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-b1664730-b7bc-4f19-ba17-1dfbfe3e610e None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:30:16.856962 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-b1664730-b7bc-4f19-ba17-1dfbfe3e610e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:30:16.864816 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1460/2916] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:30:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 17:30:16.869248 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c1d79-0ccd-40ca-91c7-db0d3293eba1 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:16.879898 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c1d79-0ccd-40ca-91c7-db0d3293eba1 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:16.884458 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd26b7d-8b7e-47f6-aa6f-056d2007d91a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:16.885151 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd26b7d-8b7e-47f6-aa6f-056d2007d91a admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:16.886439 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd26b7d-8b7e-47f6-aa6f-056d2007d91a admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:16.886743 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd26b7d-8b7e-47f6-aa6f-056d2007d91a admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:16.887779 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd26b7d-8b7e-47f6-aa6f-056d2007d91a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 17:30:16.888369 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd26b7d-8b7e-47f6-aa6f-056d2007d91a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:16.889966 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd26b7d-8b7e-47f6-aa6f-056d2007d91a admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:16.901383 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1457/2917] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 17:30:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 17:30:16.905526 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1664730-b7bc-4f19-ba17-1dfbfe3e610e None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:16.914631 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1664730-b7bc-4f19-ba17-1dfbfe3e610e None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:16.919535 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce336b4a-4aa1-402c-8cd5-49f63cb83ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:16.920191 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce336b4a-4aa1-402c-8cd5-49f63cb83ce6 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:16.920471 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce336b4a-4aa1-402c-8cd5-49f63cb83ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:16.920824 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce336b4a-4aa1-402c-8cd5-49f63cb83ce6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:16.921575 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce336b4a-4aa1-402c-8cd5-49f63cb83ce6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:16.923355 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce336b4a-4aa1-402c-8cd5-49f63cb83ce6 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:17.019554 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1461/2918] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 17:30:16 2022] POST /identity/v3/projects => generated 375 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 17:30:17.023854 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd26b7d-8b7e-47f6-aa6f-056d2007d91a admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:17.033986 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd26b7d-8b7e-47f6-aa6f-056d2007d91a admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:17.093360 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a08c7-1995-4be7-aa45-978277e94cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:17.094053 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a08c7-1995-4be7-aa45-978277e94cc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:17.094392 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a08c7-1995-4be7-aa45-978277e94cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:17.094675 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a08c7-1995-4be7-aa45-978277e94cc4 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:17.095466 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1a08c7-1995-4be7-aa45-978277e94cc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:17.101777 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1a08c7-1995-4be7-aa45-978277e94cc4 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:17.207042 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1458/2919] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 17:30:17 2022] POST /identity/v3/users => generated 288 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:30:17.211256 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce336b4a-4aa1-402c-8cd5-49f63cb83ce6 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:17.221803 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce336b4a-4aa1-402c-8cd5-49f63cb83ce6 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:17.226538 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2843b632-3556-430e-ab0a-e7232a0e1838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:17.227205 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2843b632-3556-430e-ab0a-e7232a0e1838 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:17.227512 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2843b632-3556-430e-ab0a-e7232a0e1838 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:17.227830 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2843b632-3556-430e-ab0a-e7232a0e1838 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:17.228572 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2843b632-3556-430e-ab0a-e7232a0e1838 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:17.230663 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2843b632-3556-430e-ab0a-e7232a0e1838 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:17.241196 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1462/2920] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 17:30:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 17:30:17.245862 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a08c7-1995-4be7-aa45-978277e94cc4 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:17.255895 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a08c7-1995-4be7-aa45-978277e94cc4 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:17.260772 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5a8b91-2251-41cc-9120-fc42e8d59f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:17.261448 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a8b91-2251-41cc-9120-fc42e8d59f32 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:17.261733 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a8b91-2251-41cc-9120-fc42e8d59f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:17.262009 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a8b91-2251-41cc-9120-fc42e8d59f32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:17.262737 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5a8b91-2251-41cc-9120-fc42e8d59f32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:17.264364 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5a8b91-2251-41cc-9120-fc42e8d59f32 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:17.274881 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1459/2921] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 17:30:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:30:17.278838 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2843b632-3556-430e-ab0a-e7232a0e1838 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:17.288816 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2843b632-3556-430e-ab0a-e7232a0e1838 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:17.294008 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5adc8d-b1b0-454b-9afb-c2a0d7b6bfed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:17.294882 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5adc8d-b1b0-454b-9afb-c2a0d7b6bfed admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:17.295163 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5adc8d-b1b0-454b-9afb-c2a0d7b6bfed admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:17.295442 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5adc8d-b1b0-454b-9afb-c2a0d7b6bfed admin admin] PATH_INFO: `/v3/projects/2aa3f15fdb7344eda70adba1443134ea/users/ecbc6b7c78d845ec81c3215c19e5f41b/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:17.310665 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5adc8d-b1b0-454b-9afb-c2a0d7b6bfed admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aa3f15fdb7344eda70adba1443134ea, user_id=ecbc6b7c78d845ec81c3215c19e5f41b, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:17.313386 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5adc8d-b1b0-454b-9afb-c2a0d7b6bfed admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:17.377561 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1463/2922] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 17:30:17 2022] PUT /identity/v3/projects/2aa3f15fdb7344eda70adba1443134ea/users/ecbc6b7c78d845ec81c3215c19e5f41b/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:30:17.387886 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba80b6f0-8f09-4233-95f5-c56fb1d11339 None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:17.388177 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba80b6f0-8f09-4233-95f5-c56fb1d11339 None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:17.388457 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba80b6f0-8f09-4233-95f5-c56fb1d11339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:17.415377 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-ba80b6f0-8f09-4233-95f5-c56fb1d11339 None None] MFA Rules not processed for user `ecbc6b7c78d845ec81c3215c19e5f41b`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:30:17.417221 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-ba80b6f0-8f09-4233-95f5-c56fb1d11339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:30:17.480906 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1460/2923] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:30:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:30:17.491333 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5adc8d-b1b0-454b-9afb-c2a0d7b6bfed admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:17.493041 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-1b5adc8d-b1b0-454b-9afb-c2a0d7b6bfed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:30:17.554915 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5adc8d-b1b0-454b-9afb-c2a0d7b6bfed admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:17.563548 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aefaf6-a727-4354-a6c5-9dfe6b9e4553 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:17.564221 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aefaf6-a727-4354-a6c5-9dfe6b9e4553 service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:17.564511 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aefaf6-a727-4354-a6c5-9dfe6b9e4553 service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:17.564794 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aefaf6-a727-4354-a6c5-9dfe6b9e4553 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:17.571284 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:30:17.573568 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aefaf6-a727-4354-a6c5-9dfe6b9e4553 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:17.575008 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aefaf6-a727-4354-a6c5-9dfe6b9e4553 service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:17.584460 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1464/2924] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:30:17 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:30:27.477296 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=85,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:30:27.715327 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=2 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:30:28.656331 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba80b6f0-8f09-4233-95f5-c56fb1d11339 None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:28.717243 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba80b6f0-8f09-4233-95f5-c56fb1d11339 None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:28.721835 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9689ca-a31e-4c64-8348-74490bed2dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:28.722703 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9689ca-a31e-4c64-8348-74490bed2dfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:28.722875 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9689ca-a31e-4c64-8348-74490bed2dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:28.723139 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9689ca-a31e-4c64-8348-74490bed2dfa admin admin] PATH_INFO: `/v3/users/ecbc6b7c78d845ec81c3215c19e5f41b` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:28.724279 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9689ca-a31e-4c64-8348-74490bed2dfa admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecbc6b7c78d845ec81c3215c19e5f41b)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:28.726176 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9689ca-a31e-4c64-8348-74490bed2dfa admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:28.871083 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-8e9689ca-a31e-4c64-8348-74490bed2dfa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecbc6b7c78d845ec81c3215c19e5f41b'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:28.908855 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-8e9689ca-a31e-4c64-8348-74490bed2dfa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecbc6b7c78d845ec81c3215c19e5f41b'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:28.915765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-8e9689ca-a31e-4c64-8348-74490bed2dfa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecbc6b7c78d845ec81c3215c19e5f41b'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:28.928213 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-8e9689ca-a31e-4c64-8348-74490bed2dfa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecbc6b7c78d845ec81c3215c19e5f41b'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:28.928851 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-8e9689ca-a31e-4c64-8348-74490bed2dfa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecbc6b7c78d845ec81c3215c19e5f41b'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:28.935219 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1461/2925] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 17:30:28 2022] DELETE /identity/v3/users/ecbc6b7c78d845ec81c3215c19e5f41b => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:30:28.939909 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7dcf3-0b3a-4982-bf7d-e32fdc8cd118 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:28.940390 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7dcf3-0b3a-4982-bf7d-e32fdc8cd118 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:28.940491 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7dcf3-0b3a-4982-bf7d-e32fdc8cd118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:28.958040 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-a9c7dcf3-0b3a-4982-bf7d-e32fdc8cd118 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:30:28.959243 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-a9c7dcf3-0b3a-4982-bf7d-e32fdc8cd118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:30:29.017372 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1465/2926] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:30:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:30:29.027833 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9689ca-a31e-4c64-8348-74490bed2dfa admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:29.029227 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-8e9689ca-a31e-4c64-8348-74490bed2dfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:30:29.086495 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9689ca-a31e-4c64-8348-74490bed2dfa admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:29.090719 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0d1658-687e-4c9b-ba24-cb660703c3a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:29.091360 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d1658-687e-4c9b-ba24-cb660703c3a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:29.091683 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d1658-687e-4c9b-ba24-cb660703c3a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:29.091965 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d1658-687e-4c9b-ba24-cb660703c3a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:29.101082 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0d1658-687e-4c9b-ba24-cb660703c3a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:29.102514 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0d1658-687e-4c9b-ba24-cb660703c3a3 service neutron] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:29.112300 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1462/2927] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:30:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:30:29.499757 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c7dcf3-0b3a-4982-bf7d-e32fdc8cd118 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:29.501188 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-a9c7dcf3-0b3a-4982-bf7d-e32fdc8cd118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:30:29.513667 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c7dcf3-0b3a-4982-bf7d-e32fdc8cd118 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:29.518507 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f8137-ee67-4d14-be79-ac8d377b9e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:29.519409 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f8137-ee67-4d14-be79-ac8d377b9e1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:29.519732 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f8137-ee67-4d14-be79-ac8d377b9e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:29.520009 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f8137-ee67-4d14-be79-ac8d377b9e1b admin admin] PATH_INFO: `/v3/projects/2aa3f15fdb7344eda70adba1443134ea` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:29.521227 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63f8137-ee67-4d14-be79-ac8d377b9e1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2aa3f15fdb7344eda70adba1443134ea)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:29.523137 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63f8137-ee67-4d14-be79-ac8d377b9e1b admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:29.618116 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-b63f8137-ee67-4d14-be79-ac8d377b9e1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2aa3f15fdb7344eda70adba1443134ea', 'request_id': 'req-b63f8137-ee67-4d14-be79-ac8d377b9e1b'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:29.626077 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-b63f8137-ee67-4d14-be79-ac8d377b9e1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2aa3f15fdb7344eda70adba1443134ea', 'request_id': 'req-b63f8137-ee67-4d14-be79-ac8d377b9e1b'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:29.680399 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-b63f8137-ee67-4d14-be79-ac8d377b9e1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2aa3f15fdb7344eda70adba1443134ea', 'request_id': 'req-b63f8137-ee67-4d14-be79-ac8d377b9e1b'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:29.691727 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-b63f8137-ee67-4d14-be79-ac8d377b9e1b admin admin] The token cache is being invalidate because project 2aa3f15fdb7344eda70adba1443134ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 17:30:29.696338 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-b63f8137-ee67-4d14-be79-ac8d377b9e1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:29.696338 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-b63f8137-ee67-4d14-be79-ac8d377b9e1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:29.696338 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1466/2928] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 17:30:29 2022] DELETE /identity/v3/projects/2aa3f15fdb7344eda70adba1443134ea => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:30:29.699158 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6287d3e-8651-4337-bd56-4b9801644f9e None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:29.699654 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6287d3e-8651-4337-bd56-4b9801644f9e None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:29.701746 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6287d3e-8651-4337-bd56-4b9801644f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:29.719648 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-f6287d3e-8651-4337-bd56-4b9801644f9e None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:30:29.720843 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-f6287d3e-8651-4337-bd56-4b9801644f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:30:29.737861 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:30:29.779973 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1463/2929] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:30:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:30:29.787747 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d81cf4-1a18-49df-a8fa-54b9b7dd5bc3 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:29.788075 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d81cf4-1a18-49df-a8fa-54b9b7dd5bc3 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:29.788358 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d81cf4-1a18-49df-a8fa-54b9b7dd5bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:29.807650 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-a7d81cf4-1a18-49df-a8fa-54b9b7dd5bc3 None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:30:29.809110 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-a7d81cf4-1a18-49df-a8fa-54b9b7dd5bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:30:29.816985 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1467/2930] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:30:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 17:30:29.821052 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6287d3e-8651-4337-bd56-4b9801644f9e None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:29.831728 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6287d3e-8651-4337-bd56-4b9801644f9e None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:29.836552 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0fbc4d-b768-46bb-b6b9-0fd5219e2319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:29.837609 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0fbc4d-b768-46bb-b6b9-0fd5219e2319 admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:29.837609 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0fbc4d-b768-46bb-b6b9-0fd5219e2319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:29.837809 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0fbc4d-b768-46bb-b6b9-0fd5219e2319 admin admin] PATH_INFO: `/v3/domains` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:29.839065 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0fbc4d-b768-46bb-b6b9-0fd5219e2319 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 17:30:29.839372 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0fbc4d-b768-46bb-b6b9-0fd5219e2319 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:29.841004 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0fbc4d-b768-46bb-b6b9-0fd5219e2319 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:29.852732 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1464/2931] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 18 17:30:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 17:30:29.857742 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d81cf4-1a18-49df-a8fa-54b9b7dd5bc3 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:29.867015 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d81cf4-1a18-49df-a8fa-54b9b7dd5bc3 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:29.871860 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e9a7a-05b5-4475-9260-53cdf0ce2553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:29.872511 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e9a7a-05b5-4475-9260-53cdf0ce2553 admin admin] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:29.872795 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e9a7a-05b5-4475-9260-53cdf0ce2553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:29.873068 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e9a7a-05b5-4475-9260-53cdf0ce2553 admin admin] PATH_INFO: `/v3/projects` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:29.873762 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063e9a7a-05b5-4475-9260-53cdf0ce2553 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:29.875678 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063e9a7a-05b5-4475-9260-53cdf0ce2553 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:29.947053 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1468/2932] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 18 17:30:29 2022] POST /identity/v3/projects => generated 373 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:30:29.949799 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0fbc4d-b768-46bb-b6b9-0fd5219e2319 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:29.959961 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0fbc4d-b768-46bb-b6b9-0fd5219e2319 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:30.016996 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a69bd4-44c1-45aa-a4a7-609f1ed7d568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:30.017673 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a69bd4-44c1-45aa-a4a7-609f1ed7d568 admin admin] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:30.017965 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a69bd4-44c1-45aa-a4a7-609f1ed7d568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:30.018250 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a69bd4-44c1-45aa-a4a7-609f1ed7d568 admin admin] PATH_INFO: `/v3/users` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:30.018950 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a69bd4-44c1-45aa-a4a7-609f1ed7d568 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:30.021009 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a69bd4-44c1-45aa-a4a7-609f1ed7d568 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:30.149877 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1465/2933] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 18 17:30:29 2022] POST /identity/v3/users => generated 287 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:30:30.154133 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e9a7a-05b5-4475-9260-53cdf0ce2553 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:30.164520 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e9a7a-05b5-4475-9260-53cdf0ce2553 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:30.169772 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c0626-998d-4c43-aa45-d96500e1cea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:30.170434 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c0626-998d-4c43-aa45-d96500e1cea0 admin admin] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:30.173444 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c0626-998d-4c43-aa45-d96500e1cea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:30.173444 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c0626-998d-4c43-aa45-d96500e1cea0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:30.173444 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1c0626-998d-4c43-aa45-d96500e1cea0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:30.173444 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1c0626-998d-4c43-aa45-d96500e1cea0 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:30.185561 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1469/2934] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 17:30:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:30:30.189694 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a69bd4-44c1-45aa-a4a7-609f1ed7d568 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:30.200118 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a69bd4-44c1-45aa-a4a7-609f1ed7d568 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:30.204907 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5e7978-ec45-42d6-ae58-44eaedaa629e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:30.205566 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e7978-ec45-42d6-ae58-44eaedaa629e admin admin] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:30.205872 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e7978-ec45-42d6-ae58-44eaedaa629e admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:30.206148 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e7978-ec45-42d6-ae58-44eaedaa629e admin admin] PATH_INFO: `/v3/roles` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:30.206881 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5e7978-ec45-42d6-ae58-44eaedaa629e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:30.208519 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5e7978-ec45-42d6-ae58-44eaedaa629e admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:30.218669 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1466/2935] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 18 17:30:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:30:30.222712 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c0626-998d-4c43-aa45-d96500e1cea0 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:30.233404 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c0626-998d-4c43-aa45-d96500e1cea0 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:30.237925 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a921347-2d8a-4daf-ac94-f3fda810e721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:30.238804 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a921347-2d8a-4daf-ac94-f3fda810e721 admin admin] REQUEST_METHOD: `PUT` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:30.239091 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a921347-2d8a-4daf-ac94-f3fda810e721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:30.239371 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a921347-2d8a-4daf-ac94-f3fda810e721 admin admin] PATH_INFO: `/v3/projects/d2187ae08f914172834e57319d1e4b6f/users/d42310eb10f1456992ad1f055fcefe8e/roles/aa0552e8ba924f5d9cb1740da7d4a4dc` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:30.254744 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a921347-2d8a-4daf-ac94-f3fda810e721 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2187ae08f914172834e57319d1e4b6f, user_id=d42310eb10f1456992ad1f055fcefe8e, role_id=aa0552e8ba924f5d9cb1740da7d4a4dc)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:30.257416 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a921347-2d8a-4daf-ac94-f3fda810e721 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:30.310063 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1470/2936] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 18 17:30:30 2022] PUT /identity/v3/projects/d2187ae08f914172834e57319d1e4b6f/users/d42310eb10f1456992ad1f055fcefe8e/roles/aa0552e8ba924f5d9cb1740da7d4a4dc => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:30:30.320667 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8798ca7a-1547-4b65-9955-6811747f031d None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:30.322453 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8798ca7a-1547-4b65-9955-6811747f031d None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:30.322453 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8798ca7a-1547-4b65-9955-6811747f031d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:30.348455 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-8798ca7a-1547-4b65-9955-6811747f031d None None] MFA Rules not processed for user `d42310eb10f1456992ad1f055fcefe8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:30:30.349674 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-8798ca7a-1547-4b65-9955-6811747f031d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:30:30.415605 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1467/2937] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:30:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:30:30.427365 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a921347-2d8a-4daf-ac94-f3fda810e721 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:30.432862 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-4a921347-2d8a-4daf-ac94-f3fda810e721 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:30:30.451057 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:30:30.486597 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a921347-2d8a-4daf-ac94-f3fda810e721 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:30.490384 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ef788b-28d4-4664-b73e-f1f298d8f5d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f2e55df807c43d7951fa4be67ef5e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:30.491047 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef788b-28d4-4664-b73e-f1f298d8f5d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:30.495930 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef788b-28d4-4664-b73e-f1f298d8f5d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:30.495930 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef788b-28d4-4664-b73e-f1f298d8f5d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:30.499980 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ef788b-28d4-4664-b73e-f1f298d8f5d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:30.501405 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ef788b-28d4-4664-b73e-f1f298d8f5d1 service cinder] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:30.510236 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1471/2938] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:30:30 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 17:30:32.263769 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0e08f-d062-470f-817b-428b4e4c4b7c None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:32.264269 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0e08f-d062-470f-817b-428b4e4c4b7c None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:32.264365 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0e08f-d062-470f-817b-428b4e4c4b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:32.282073 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-b4d0e08f-d062-470f-817b-428b4e4c4b7c None None] MFA Rules not processed for user `d42310eb10f1456992ad1f055fcefe8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:30:32.283251 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-b4d0e08f-d062-470f-817b-428b4e4c4b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:30:32.290023 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1468/2939] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:30:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:30:32.326810 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ef788b-28d4-4664-b73e-f1f298d8f5d1 service cinder] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:32.328258 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-01ef788b-28d4-4664-b73e-f1f298d8f5d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:30:32.405278 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ef788b-28d4-4664-b73e-f1f298d8f5d1 service cinder] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:32.408868 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd6425f-8608-4d6e-b274-24ff6c8f2bc0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0887908bcb0d4072bf6acb4cbaf1af83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:32.409505 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd6425f-8608-4d6e-b274-24ff6c8f2bc0 service glance] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:32.409784 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd6425f-8608-4d6e-b274-24ff6c8f2bc0 service glance] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:32.410054 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd6425f-8608-4d6e-b274-24ff6c8f2bc0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:32.411811 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-4dd6425f-8608-4d6e-b274-24ff6c8f2bc0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:30:32.500901 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd6425f-8608-4d6e-b274-24ff6c8f2bc0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:32.502348 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd6425f-8608-4d6e-b274-24ff6c8f2bc0 service glance] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:32.523531 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1472/2940] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:30:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:30:32.787277 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d0e08f-d062-470f-817b-428b4e4c4b7c None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:32.788677 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-b4d0e08f-d062-470f-817b-428b4e4c4b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:30:32.801664 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:30:32.862148 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d0e08f-d062-470f-817b-428b4e4c4b7c None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:32.867201 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aad454-96da-4b0d-b903-5be8ace1fc2e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70f704a231a5474e8067a6c2dd88b355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:32.867955 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aad454-96da-4b0d-b903-5be8ace1fc2e service placement] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:32.868526 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aad454-96da-4b0d-b903-5be8ace1fc2e service placement] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:32.869029 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aad454-96da-4b0d-b903-5be8ace1fc2e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:32.871787 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-17aad454-96da-4b0d-b903-5be8ace1fc2e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:30:32.883664 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aad454-96da-4b0d-b903-5be8ace1fc2e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:32.885249 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aad454-96da-4b0d-b903-5be8ace1fc2e service placement] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:32.896279 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1469/2941] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:30:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:30:42.520463 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=92 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:30:42.891921 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=27 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:30:48.961005 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b340a-6293-4cfb-b8e1-07c42317e0de None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:48.961441 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b340a-6293-4cfb-b8e1-07c42317e0de None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:48.961618 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b340a-6293-4cfb-b8e1-07c42317e0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:48.980451 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-4f0b340a-6293-4cfb-b8e1-07c42317e0de None None] MFA Rules not processed for user `da799b0044f54da6835aa06e43ce1a6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:30:48.982124 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-4f0b340a-6293-4cfb-b8e1-07c42317e0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:30:49.037967 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1473/2942] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jul 18 17:30:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:30:49.181962 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aad454-96da-4b0d-b903-5be8ace1fc2e service placement] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:49.183394 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-17aad454-96da-4b0d-b903-5be8ace1fc2e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:30:49.253704 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aad454-96da-4b0d-b903-5be8ace1fc2e service placement] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:49.258684 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf5260-7177-4077-93ea-38719d4241e5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43854d7a6df2466eb7976fabe6011767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:49.259323 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf5260-7177-4077-93ea-38719d4241e5 service swift] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:49.259645 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf5260-7177-4077-93ea-38719d4241e5 service swift] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:49.259921 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf5260-7177-4077-93ea-38719d4241e5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:49.268338 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaf5260-7177-4077-93ea-38719d4241e5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:49.269972 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaf5260-7177-4077-93ea-38719d4241e5 service swift] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:49.278728 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1470/2943] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jul 18 17:30:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 17:30:50.053314 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc245cd-b651-436c-b35b-ff67e034bce1 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:50.053751 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc245cd-b651-436c-b35b-ff67e034bce1 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:50.053901 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc245cd-b651-436c-b35b-ff67e034bce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:50.114842 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-5bc245cd-b651-436c-b35b-ff67e034bce1 None None] MFA Rules not processed for user `da799b0044f54da6835aa06e43ce1a6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:30:50.116113 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-5bc245cd-b651-436c-b35b-ff67e034bce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:30:50.132845 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1474/2944] 10.222.0.26 () {58 vars in 967 bytes} [Mon Jul 18 17:30:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:30:50.148171 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf5260-7177-4077-93ea-38719d4241e5 service swift] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:50.167205 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf5260-7177-4077-93ea-38719d4241e5 service swift] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:50.172598 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d1fd2-0c34-4d46-b0e9-5f076c950ac6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43854d7a6df2466eb7976fabe6011767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:50.173238 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d1fd2-0c34-4d46-b0e9-5f076c950ac6 service swift] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:50.173633 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d1fd2-0c34-4d46-b0e9-5f076c950ac6 service swift] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:50.173921 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d1fd2-0c34-4d46-b0e9-5f076c950ac6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:50.182868 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980d1fd2-0c34-4d46-b0e9-5f076c950ac6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:50.185008 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980d1fd2-0c34-4d46-b0e9-5f076c950ac6 service swift] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:50.193490 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1471/2945] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jul 18 17:30:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 17:30:50.725862 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd856622-41c4-4011-ac53-6ab456260935 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:50.726154 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd856622-41c4-4011-ac53-6ab456260935 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:50.726436 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd856622-41c4-4011-ac53-6ab456260935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:50.744939 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-bd856622-41c4-4011-ac53-6ab456260935 None None] MFA Rules not processed for user `da799b0044f54da6835aa06e43ce1a6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:30:50.746478 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-bd856622-41c4-4011-ac53-6ab456260935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:30:50.753656 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1475/2946] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jul 18 17:30:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:30:50.765512 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d1fd2-0c34-4d46-b0e9-5f076c950ac6 service swift] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:50.775993 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d1fd2-0c34-4d46-b0e9-5f076c950ac6 service swift] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:50.781439 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ef80b0-d992-4e37-b809-fae8c1fad2d2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43854d7a6df2466eb7976fabe6011767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:50.782280 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef80b0-d992-4e37-b809-fae8c1fad2d2 service swift] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:50.782518 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef80b0-d992-4e37-b809-fae8c1fad2d2 service swift] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:50.782653 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef80b0-d992-4e37-b809-fae8c1fad2d2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:50.790733 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ef80b0-d992-4e37-b809-fae8c1fad2d2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:50.792337 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ef80b0-d992-4e37-b809-fae8c1fad2d2 service swift] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:50.801089 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1472/2947] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jul 18 17:30:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 17:30:51.337084 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893476e0-f91c-44d5-ac93-04d7fd1334b6 None None] REQUEST_METHOD: `POST` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:51.337523 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893476e0-f91c-44d5-ac93-04d7fd1334b6 None None] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:51.337617 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893476e0-f91c-44d5-ac93-04d7fd1334b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:51.356810 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.auth.core [None req-893476e0-f91c-44d5-ac93-04d7fd1334b6 None None] MFA Rules not processed for user `da799b0044f54da6835aa06e43ce1a6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:30:51.358014 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-893476e0-f91c-44d5-ac93-04d7fd1334b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:30:51.365366 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1476/2948] 10.222.0.26 () {58 vars in 967 bytes} [Mon Jul 18 17:30:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:30:51.378456 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ef80b0-d992-4e37-b809-fae8c1fad2d2 service swift] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:51.389947 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ef80b0-d992-4e37-b809-fae8c1fad2d2 service swift] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:51.394834 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa48847-61b1-45ec-b94a-2a4b5c23512c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43854d7a6df2466eb7976fabe6011767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:51.395455 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa48847-61b1-45ec-b94a-2a4b5c23512c service swift] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:51.395768 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa48847-61b1-45ec-b94a-2a4b5c23512c service swift] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:51.396040 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa48847-61b1-45ec-b94a-2a4b5c23512c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:51.404326 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa48847-61b1-45ec-b94a-2a4b5c23512c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:51.405883 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa48847-61b1-45ec-b94a-2a4b5c23512c service swift] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:51.413781 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1473/2949] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jul 18 17:30:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 17:30:54.034927 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893476e0-f91c-44d5-ac93-04d7fd1334b6 None None] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:54.099018 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893476e0-f91c-44d5-ac93-04d7fd1334b6 None None] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:54.103538 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92e0f20-42fd-43e7-a765-7722eb7825a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:54.104414 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e0f20-42fd-43e7-a765-7722eb7825a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:54.104700 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e0f20-42fd-43e7-a765-7722eb7825a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:54.104979 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e0f20-42fd-43e7-a765-7722eb7825a1 admin admin] PATH_INFO: `/v3/users/d42310eb10f1456992ad1f055fcefe8e` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:54.106399 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92e0f20-42fd-43e7-a765-7722eb7825a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d42310eb10f1456992ad1f055fcefe8e)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:54.108417 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92e0f20-42fd-43e7-a765-7722eb7825a1 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:54.258825 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:30:54.260352 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-d92e0f20-42fd-43e7-a765-7722eb7825a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd42310eb10f1456992ad1f055fcefe8e'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:54.337439 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-d92e0f20-42fd-43e7-a765-7722eb7825a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd42310eb10f1456992ad1f055fcefe8e'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:54.338082 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-d92e0f20-42fd-43e7-a765-7722eb7825a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd42310eb10f1456992ad1f055fcefe8e'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:54.338690 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-d92e0f20-42fd-43e7-a765-7722eb7825a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd42310eb10f1456992ad1f055fcefe8e'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:54.346257 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-d92e0f20-42fd-43e7-a765-7722eb7825a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd42310eb10f1456992ad1f055fcefe8e'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:54.370532 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1477/2950] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 17:30:54 2022] DELETE /identity/v3/users/d42310eb10f1456992ad1f055fcefe8e => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:30:54.370733 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b2c2b-c1f2-4543-86db-b9ad8adf6b9d None None] REQUEST_METHOD: `POST` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:54.370733 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b2c2b-c1f2-4543-86db-b9ad8adf6b9d None None] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:54.370733 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b2c2b-c1f2-4543-86db-b9ad8adf6b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:54.388463 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.auth.core [None req-4a0b2c2b-c1f2-4543-86db-b9ad8adf6b9d None None] MFA Rules not processed for user `e18df28d16ba462694258c7ad963170a`. Rule list: `[]` (Enabled: `True`). {{(pid=80368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:30:54.389669 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-4a0b2c2b-c1f2-4543-86db-b9ad8adf6b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:30:54.451751 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1474/2951] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 18 17:30:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:30:54.461474 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92e0f20-42fd-43e7-a765-7722eb7825a1 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:54.462851 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-d92e0f20-42fd-43e7-a765-7722eb7825a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:30:54.518910 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92e0f20-42fd-43e7-a765-7722eb7825a1 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:54.522446 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc773e35-1beb-4e23-9157-55f66fd4fc5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:54.523085 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc773e35-1beb-4e23-9157-55f66fd4fc5f service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:54.523365 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc773e35-1beb-4e23-9157-55f66fd4fc5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:54.523670 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc773e35-1beb-4e23-9157-55f66fd4fc5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:54.532819 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc773e35-1beb-4e23-9157-55f66fd4fc5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:54.534229 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc773e35-1beb-4e23-9157-55f66fd4fc5f service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:54.543603 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1478/2952] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:30:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:30:54.786745 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b2c2b-c1f2-4543-86db-b9ad8adf6b9d None None] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:30:54.788947 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-4a0b2c2b-c1f2-4543-86db-b9ad8adf6b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:30:54.807570 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b2c2b-c1f2-4543-86db-b9ad8adf6b9d None None] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:30:54.812898 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0082430e-341b-4597-82d2-d9f796a70c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:30:54.813765 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0082430e-341b-4597-82d2-d9f796a70c73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:30:54.814049 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0082430e-341b-4597-82d2-d9f796a70c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:30:54.814405 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0082430e-341b-4597-82d2-d9f796a70c73 admin admin] PATH_INFO: `/v3/projects/d2187ae08f914172834e57319d1e4b6f` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:30:54.815565 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0082430e-341b-4597-82d2-d9f796a70c73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2187ae08f914172834e57319d1e4b6f)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:30:54.817531 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0082430e-341b-4597-82d2-d9f796a70c73 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:30:54.895850 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:30:54.904988 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-0082430e-341b-4597-82d2-d9f796a70c73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2187ae08f914172834e57319d1e4b6f', 'request_id': 'req-0082430e-341b-4597-82d2-d9f796a70c73'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:54.910265 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-0082430e-341b-4597-82d2-d9f796a70c73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2187ae08f914172834e57319d1e4b6f', 'request_id': 'req-0082430e-341b-4597-82d2-d9f796a70c73'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:54.916387 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-0082430e-341b-4597-82d2-d9f796a70c73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2187ae08f914172834e57319d1e4b6f', 'request_id': 'req-0082430e-341b-4597-82d2-d9f796a70c73'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:54.971358 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-0082430e-341b-4597-82d2-d9f796a70c73 admin admin] The token cache is being invalidate because project d2187ae08f914172834e57319d1e4b6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 17:30:54.971682 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-0082430e-341b-4597-82d2-d9f796a70c73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:54.972318 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-0082430e-341b-4597-82d2-d9f796a70c73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:30:54.973867 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1475/2953] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 17:30:54 2022] DELETE /identity/v3/projects/d2187ae08f914172834e57319d1e4b6f => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:31:04.539321 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=39,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:31:04.977960 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=11,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:31:21.057400 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc773e35-1beb-4e23-9157-55f66fd4fc5f service neutron] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:31:21.059013 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-cc773e35-1beb-4e23-9157-55f66fd4fc5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:31:21.070456 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc773e35-1beb-4e23-9157-55f66fd4fc5f service neutron] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:31:21.119888 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f9a313-7a35-4a71-8dc2-ebd41a1c1f9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:31:21.120541 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9a313-7a35-4a71-8dc2-ebd41a1c1f9e service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:31:21.120860 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9a313-7a35-4a71-8dc2-ebd41a1c1f9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:31:21.121154 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9a313-7a35-4a71-8dc2-ebd41a1c1f9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:31:21.122906 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-70f9a313-7a35-4a71-8dc2-ebd41a1c1f9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:31:21.179755 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f9a313-7a35-4a71-8dc2-ebd41a1c1f9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:31:21.181186 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f9a313-7a35-4a71-8dc2-ebd41a1c1f9e service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:31:21.190908 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1479/2954] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:31:21 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:31:29.686782 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0082430e-341b-4597-82d2-d9f796a70c73 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:31:29.688296 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-0082430e-341b-4597-82d2-d9f796a70c73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:31:29.763659 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0082430e-341b-4597-82d2-d9f796a70c73 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:31:29.770448 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800c5b8e-dc36-4eb5-a387-f0f9e42f7397 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b868bfa44b94ca5b470166d39f20e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:31:29.771109 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800c5b8e-dc36-4eb5-a387-f0f9e42f7397 service nova] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:31:29.771393 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800c5b8e-dc36-4eb5-a387-f0f9e42f7397 service nova] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:31:29.771702 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800c5b8e-dc36-4eb5-a387-f0f9e42f7397 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:31:29.773493 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-800c5b8e-dc36-4eb5-a387-f0f9e42f7397 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:31:29.786614 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800c5b8e-dc36-4eb5-a387-f0f9e42f7397 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:31:29.788258 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800c5b8e-dc36-4eb5-a387-f0f9e42f7397 service nova] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:31:29.799755 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1476/2955] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:31:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:31:31.186878 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=48 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:31:35.708887 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f9a313-7a35-4a71-8dc2-ebd41a1c1f9e service neutron] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:31:35.710402 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-70f9a313-7a35-4a71-8dc2-ebd41a1c1f9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:31:35.721952 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f9a313-7a35-4a71-8dc2-ebd41a1c1f9e service neutron] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:31:35.725641 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d167a9dc-272d-4a06-bf85-0f5491447dc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a6054dbfbbc487794d2a32061b0ac82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:31:35.726285 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167a9dc-272d-4a06-bf85-0f5491447dc8 service neutron] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:31:35.726572 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167a9dc-272d-4a06-bf85-0f5491447dc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:31:35.726874 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167a9dc-272d-4a06-bf85-0f5491447dc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:31:35.728614 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-d167a9dc-272d-4a06-bf85-0f5491447dc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:31:35.791741 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d167a9dc-272d-4a06-bf85-0f5491447dc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:31:35.793178 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d167a9dc-272d-4a06-bf85-0f5491447dc8 service neutron] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:31:35.803537 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1480/2956] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:31:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:31:39.794289 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=34 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:31:42.763125 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800c5b8e-dc36-4eb5-a387-f0f9e42f7397 service nova] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:31:42.765961 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-800c5b8e-dc36-4eb5-a387-f0f9e42f7397 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:31:42.851199 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800c5b8e-dc36-4eb5-a387-f0f9e42f7397 service nova] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:31:42.856508 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89cbac4-3477-4d4e-9f1a-bc54bddc50e6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70f704a231a5474e8067a6c2dd88b355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:31:42.857367 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89cbac4-3477-4d4e-9f1a-bc54bddc50e6 service placement] REQUEST_METHOD: `GET` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:31:42.857725 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89cbac4-3477-4d4e-9f1a-bc54bddc50e6 service placement] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:31:42.858099 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89cbac4-3477-4d4e-9f1a-bc54bddc50e6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:31:42.861687 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-b89cbac4-3477-4d4e-9f1a-bc54bddc50e6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:31:42.874145 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89cbac4-3477-4d4e-9f1a-bc54bddc50e6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:31:42.876559 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89cbac4-3477-4d4e-9f1a-bc54bddc50e6 service placement] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:31:42.887324 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1477/2957] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:31:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:31:42.909622 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d167a9dc-272d-4a06-bf85-0f5491447dc8 service neutron] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:31:42.911305 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-d167a9dc-272d-4a06-bf85-0f5491447dc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:31:42.926492 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d167a9dc-272d-4a06-bf85-0f5491447dc8 service neutron] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:31:42.931410 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf3c071-010f-443c-8372-6c4ce194325b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:31:42.932397 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf3c071-010f-443c-8372-6c4ce194325b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:31:42.932648 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf3c071-010f-443c-8372-6c4ce194325b admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:31:42.933121 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf3c071-010f-443c-8372-6c4ce194325b admin admin] PATH_INFO: `/v3/users/e1fc02859caf4ac09a85a4e5f962c14b` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:31:42.934115 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf3c071-010f-443c-8372-6c4ce194325b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1fc02859caf4ac09a85a4e5f962c14b)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:31:42.938112 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf3c071-010f-443c-8372-6c4ce194325b admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:31:43.095269 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-faf3c071-010f-443c-8372-6c4ce194325b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1fc02859caf4ac09a85a4e5f962c14b'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:31:43.143471 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-faf3c071-010f-443c-8372-6c4ce194325b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1fc02859caf4ac09a85a4e5f962c14b'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:31:43.144065 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-faf3c071-010f-443c-8372-6c4ce194325b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1fc02859caf4ac09a85a4e5f962c14b'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:31:43.144677 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-faf3c071-010f-443c-8372-6c4ce194325b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1fc02859caf4ac09a85a4e5f962c14b'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:31:43.151870 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-faf3c071-010f-443c-8372-6c4ce194325b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1fc02859caf4ac09a85a4e5f962c14b'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:31:43.169661 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1481/2958] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 17:31:42 2022] DELETE /identity/v3/users/e1fc02859caf4ac09a85a4e5f962c14b => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:31:43.402091 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89cbac4-3477-4d4e-9f1a-bc54bddc50e6 service placement] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:31:43.413867 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89cbac4-3477-4d4e-9f1a-bc54bddc50e6 service placement] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:31:43.472887 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec7563b-db90-4e7e-a255-1df567310660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:31:43.473739 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec7563b-db90-4e7e-a255-1df567310660 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:31:43.474025 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec7563b-db90-4e7e-a255-1df567310660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:31:43.474305 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec7563b-db90-4e7e-a255-1df567310660 admin admin] PATH_INFO: `/v3/projects/e4afeb76581942e4a0d72e4568907201` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:31:43.475515 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec7563b-db90-4e7e-a255-1df567310660 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4afeb76581942e4a0d72e4568907201)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:31:43.477411 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec7563b-db90-4e7e-a255-1df567310660 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:31:43.579509 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-7ec7563b-db90-4e7e-a255-1df567310660 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4afeb76581942e4a0d72e4568907201', 'request_id': 'req-7ec7563b-db90-4e7e-a255-1df567310660'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:31:43.585196 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-7ec7563b-db90-4e7e-a255-1df567310660 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4afeb76581942e4a0d72e4568907201', 'request_id': 'req-7ec7563b-db90-4e7e-a255-1df567310660'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:31:43.591606 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-7ec7563b-db90-4e7e-a255-1df567310660 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4afeb76581942e4a0d72e4568907201', 'request_id': 'req-7ec7563b-db90-4e7e-a255-1df567310660'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:31:43.685656 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-7ec7563b-db90-4e7e-a255-1df567310660 admin admin] The token cache is being invalidate because project e4afeb76581942e4a0d72e4568907201 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 17:31:43.685876 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-7ec7563b-db90-4e7e-a255-1df567310660 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:31:43.686525 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-7ec7563b-db90-4e7e-a255-1df567310660 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:31:43.688978 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1478/2959] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 17:31:43 2022] DELETE /identity/v3/projects/e4afeb76581942e4a0d72e4568907201 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 17:31:43.693204 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf3c071-010f-443c-8372-6c4ce194325b admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:31:43.694599 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-faf3c071-010f-443c-8372-6c4ce194325b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:31:43.708177 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf3c071-010f-443c-8372-6c4ce194325b admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:31:43.767704 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4df79e4-dfbd-4b87-abfa-9503fb2d7b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:31:43.768577 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df79e4-dfbd-4b87-abfa-9503fb2d7b95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:31:43.768855 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df79e4-dfbd-4b87-abfa-9503fb2d7b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:31:43.769128 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df79e4-dfbd-4b87-abfa-9503fb2d7b95 admin admin] PATH_INFO: `/v3/users/0f52c81946ee49bb805781b3e69e4bbb` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:31:43.770305 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4df79e4-dfbd-4b87-abfa-9503fb2d7b95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f52c81946ee49bb805781b3e69e4bbb)` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:31:43.772246 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4df79e4-dfbd-4b87-abfa-9503fb2d7b95 admin admin] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:31:43.777455 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:31:43.980345 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-e4df79e4-dfbd-4b87-abfa-9503fb2d7b95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f52c81946ee49bb805781b3e69e4bbb'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:31:44.051956 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-e4df79e4-dfbd-4b87-abfa-9503fb2d7b95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f52c81946ee49bb805781b3e69e4bbb'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:31:44.052582 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-e4df79e4-dfbd-4b87-abfa-9503fb2d7b95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f52c81946ee49bb805781b3e69e4bbb'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:31:44.053188 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-e4df79e4-dfbd-4b87-abfa-9503fb2d7b95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f52c81946ee49bb805781b3e69e4bbb'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:31:44.060275 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.notifications [None req-e4df79e4-dfbd-4b87-abfa-9503fb2d7b95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f52c81946ee49bb805781b3e69e4bbb'} {{(pid=80367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:31:44.079341 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1482/2960] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 18 17:31:43 2022] DELETE /identity/v3/users/0f52c81946ee49bb805781b3e69e4bbb => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:31:44.328033 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec7563b-db90-4e7e-a255-1df567310660 admin admin] Authenticating user token {{(pid=80368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:31:44.332220 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.fernet_utils [None req-7ec7563b-db90-4e7e-a255-1df567310660 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:31:44.342825 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec7563b-db90-4e7e-a255-1df567310660 admin admin] Validating token access rules against request {{(pid=80368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:31:44.360077 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:31:44.400159 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d8d839-ac97-48bf-8a9e-41259ec3b257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e18df28d16ba462694258c7ad963170a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e08d80b293148beac001ab7796d9842', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:31:44.401378 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d8d839-ac97-48bf-8a9e-41259ec3b257 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:31:44.401670 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d8d839-ac97-48bf-8a9e-41259ec3b257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:31:44.401961 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d8d839-ac97-48bf-8a9e-41259ec3b257 admin admin] PATH_INFO: `/v3/projects/ae03122aa4c944249656119878e22d25` {{(pid=80368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:31:44.403157 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d8d839-ac97-48bf-8a9e-41259ec3b257 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae03122aa4c944249656119878e22d25)` {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:31:44.405104 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d8d839-ac97-48bf-8a9e-41259ec3b257 admin admin] RBAC: Authorization granted {{(pid=80368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:31:44.501129 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-86d8d839-ac97-48bf-8a9e-41259ec3b257 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae03122aa4c944249656119878e22d25', 'request_id': 'req-86d8d839-ac97-48bf-8a9e-41259ec3b257'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:31:44.506819 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-86d8d839-ac97-48bf-8a9e-41259ec3b257 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae03122aa4c944249656119878e22d25', 'request_id': 'req-86d8d839-ac97-48bf-8a9e-41259ec3b257'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:31:44.513169 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-86d8d839-ac97-48bf-8a9e-41259ec3b257 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae03122aa4c944249656119878e22d25', 'request_id': 'req-86d8d839-ac97-48bf-8a9e-41259ec3b257'} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:31:44.554587 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-86d8d839-ac97-48bf-8a9e-41259ec3b257 admin admin] The token cache is being invalidate because project ae03122aa4c944249656119878e22d25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 17:31:44.554879 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-86d8d839-ac97-48bf-8a9e-41259ec3b257 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:31:44.555545 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG keystone.notifications [None req-86d8d839-ac97-48bf-8a9e-41259ec3b257 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:31:44.557113 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: [pid: 80368|app: 0|req: 1479/2961] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 18 17:31:44 2022] DELETE /identity/v3/projects/ae03122aa4c944249656119878e22d25 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:31:54.074936 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:31:54.589936 dv-cinder-846172-3-iscsi devstack@keystone.service[80368]: DEBUG dbcounter [-] [80368] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:35:39.779995 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4df79e4-dfbd-4b87-abfa-9503fb2d7b95 admin admin] Authenticating user token {{(pid=80367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:35:39.781382 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-e4df79e4-dfbd-4b87-abfa-9503fb2d7b95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:35:39.926938 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4df79e4-dfbd-4b87-abfa-9503fb2d7b95 admin admin] Validating token access rules against request {{(pid=80367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:35:39.987118 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76a1a36-271d-4aa5-affb-f5b868a4f420 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70f704a231a5474e8067a6c2dd88b355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb9747e2299c4e0991fd7edb4462c7e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:35:39.987803 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a1a36-271d-4aa5-affb-f5b868a4f420 service placement] REQUEST_METHOD: `GET` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:35:39.988088 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a1a36-271d-4aa5-affb-f5b868a4f420 service placement] SCRIPT_NAME: `/identity` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:35:39.988361 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a1a36-271d-4aa5-affb-f5b868a4f420 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:35:39.990151 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.fernet_utils [None req-f76a1a36-271d-4aa5-affb-f5b868a4f420 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:35:40.001474 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76a1a36-271d-4aa5-affb-f5b868a4f420 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:35:40.003047 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76a1a36-271d-4aa5-affb-f5b868a4f420 service placement] RBAC: Authorization granted {{(pid=80367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:35:40.012960 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: [pid: 80367|app: 0|req: 1483/2962] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 18 17:35:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:35:50.008903 dv-cinder-846172-3-iscsi devstack@keystone.service[80367]: DEBUG dbcounter [-] [80367] Writing DB stats keystone:SELECT=89 {{(pid=80367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}